python - Redis - Parse data stream provided by remote server -
i have set locally redis , want connect remote server provides synthetic data stream in form < id, value >. have far managed connect server mentioned above using sockets, read data stream , print it. instead want store pairs in hash data structure (i'm going store more informations each id later). problem don't know how parse data stream in order use hget
, how continuously. in higher level able pass name , value incoming data stream arguments hget
. forgot mention i'm using python api. far:
import socket client_socket = socket.socket(socket.af_inet, socket.sock_stream) client_socket.connect(('xx.xx.xx.xxx', 1337)) while 1: data = client_socket.recv(512) print data
a sample of data stream:
'amzn,780.6758\n' 'tsla,197.1802\n' 'csco,29.7491\n' 'goog,761.3758\n' 'aapl,112.4122\n' 'grpn,4.5848\n' 'fb,121.1232\n' 'msft,60.3529\n' 'intc,35.9056\n' 'nvda,94.473\n' 'qcom,68.7389\n' 'amzn,780.6761\n' 'tsla,197.1798\n' 'csco,29.7486\n' 'goog,761.3755\n' 'aapl,112.4122\n' 'grpn,4.5848\n' 'fb,121.1237\n' 'msft,60.353\n' 'intc,35.9054\n' 'nvda,94.473\n' 'qcom,68.7391\n'
i'm not sure if there guarantee lines formatted let's are.
parsing single non-empty line key/value pair simple :
key, value = line.strip().split(",", 1)
assuming data may incomplete (unterminated record) , it's linefeed marks end of record, can store incomplete records in buffer , add them before parsing, function might this:
def run(client_socket): buffer = "" while true: data = client_socket.recv(512) # not sure following lines makes sense - # may want handle exceptions # or whatever if not data: break # add buffer data = buffer + data # split on newlines lines = data.splitlines() # check if have incomplete record # (if doesn't end newline) if data[-1] != '\n': # incomplete record, store # process next time buffer = lines.pop() else: # records complete call, # empty buffer next turn buffer = "" # handle our records: line in filter(none, lines): k, v = line.split(",", 1) do_something_with(k, v)
the implementation of do_something_with(k, v)
left exercise reader.
Comments
Post a Comment