api - Search/Filter/Select/Manipulate data from a website using Python -
i'm working on project requires me go website, pick search mode (name, year, number, etc), search name, select amongst results specific type (filtering in other words), pick option save results opposed emailing them, pick format save them download them clicking save button.
my question is, there way steps using python program? aware of extracting data , downloading pages/images, wondering if there way write script manipulate data, , person manually do, large number of iterations.
i've thought of looking url structures, , finding way generate each iteration accurate url, if works, i'm still stuck because of "save" button, can't find link automatically download data want, , using function of urllib2 library download page not actual file want.
any idea on how approach this? reference/tutorial extremely helpful, thanks!
edit: when inspect save button here get: search button
this depend lot on website targeting , how search implemented.
for websites, reddit, have open api can add .json
extension url , json string response opposed pure html.
for using rest api or json response, can load python dictionary using json module this
import json json_response = '{"customers":[{"name":"carlos", "age":4}, {"name":"jim", "age":5}]}' rdict = json.loads(json_response) def print_names(data): entry in data["customers"]: print(entry["name"]) print_names(rdict)
Comments
Post a Comment