how to write Spark data frame to Neo4j database -


i'd build workflow:

  • preprocess data spark, ending data frame
  • write such dataframe neo4j set of nodes

my idea basic: write each row in df node, each column value represents value of node's attribute

i have seen many articles, including neo4j-spark-connector , introducing neo4j 3.0 apache spark connector focus on importing spark data neo4j db... far, wasn't able find clear example of writing spark data frame neo4j database.

any pointer documentation or basic examples appreciated.

you can write routine , use opensource neo4j java driver

https://github.com/neo4j/neo4j-java-driver

for example.

simple serialise result of rdd (using rdd.tojson) , use above driver create neo4j nodes , push neo4j instance.


-

Comments

Post a Comment

Popular posts from this blog

.net - Bulk insert via Dapper is slower than inserting rows one-by-one -

i'm using dapper insert data realtime feed sql server, care performance. recently, noticed strange. out of box, if give dapper collection , insert query, fires insert statements each element. tests show can insert 1800 objects 12 fields in 1 second way (counting connection.execute(...) running time. now, didn't find batch insert functionality in dapper , implemented own (constructing parameter list , sql query). after that, found out can insert 1 batch in 3 seconds (which limited 1000 rows) (again, counting connection.execute(...) calls. so, makes batching 6 times slower sending each row in separate query. can explain me? thought people use batch operations speed process. insert time 1 second @ most. use sql server 2012 standard on local network. table i'm inserting has clustered index on primary key (which bigint field), no non-clustered indexes , triggers. can post code, there's nothing special i'm not sure why using dapper execute extension ...
Read more

shared memory - gstreamer shmsrc and shmsink with h264 data -

i trying share h264 encoded data gstreamer 2 processes(both based on gstreamer).after research way found use shm plugin. trying do gstreamer--->h264 encoder--->shmsink shmrc--->process1 shmrc--->process2 i able raw data videotestsrc , webcam working. h264 encoded data doesn't. test pipeline gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,format=yuy2 ! x264enc ! shmsink socket-path=/tmp/foo sync=true wait-for- connection=false shm-size=10000000 gst-launch-1.0 shmsrc socket-path=/tmp/foo ! avdec_h264 ! video/x- raw,width=640,height=480,framerate=25/1,format=yuy2 ! autovideosink have tried shm plugins h264 encoded data, please help iam not aware of capabilities of sink used in "autovideosink", per knowledge either need use videoconvert if format supported sink (like kmssink or ximagesink) different provided source (in case yuy2) or use videoparse if camera format supported sink. may check using gst-inspect-1...
Read more

java - is not an enclosing class / new Intent Cannot Resolve Constructor -

thanks checking out question! i have relativly ismple loop add onclicklistener each imageview add layout, when trying add new intent it, gives me 1 of following errors: string[] imageurls = imageurlsstring.split("/"); (int = 0; < imageurls.length; i++){ imageview image = new imageview(this); image.setlayoutparams(new android.view.viewgroup.layoutparams(getpx(182),getpx(256))); image.setpadding(getpx(3),getpx(3),getpx(3),getpx(3)); final string imageurl = ".../images/" + imageurls[i]; picasso.with(this).load(imageurl).into(image); image.setonclicklistener(new view.onclicklistener() { @override public void onclick(view view) { intent intent; intent = new intent(this, imageactivity.class); intent.putextra("url", imageurl); startactivity(intent); } }); linearlayout.addview(image)...
Read more