performance - apache phoenix, slow write speed when change currentSCN -


have old hbase table (billion records) , need use data create new phoenix table. want preserve record modtime old record. based on phoenix suggestion. give property currentscn when connect. seems slow. while doc

keep in mind creating new connection not expensive operation. same underlying hconnection used connections same cluster, it’s more or less instantiating few objects.

i test using simple program.

        class.forname("org.apache.phoenix.jdbc.phoenixdriver");         int x = 0;         for(int i=1000000 ;i< 2000000;i++ ) {             properties prop = new properties();             prop.setproperty("currentscn",string.valueof(i));             connection con = drivermanager.getconnection("jdbc:phoenix:a1.cluster",prop);             con.setautocommit(false);              preparedstatement  stmt2 = con.preparestatement("upsert fff values (?,?,?,?,?)");             stmt2.setstring(1,string.valueof(i));             stmt2.settimestamp(2, timestamp.from(instant.ofepochmilli(system.currenttimemillis())));             stmt2.setstring(3, null);             stmt2.setdouble(4, double.valueof(i));             stmt2.setstring(5, null);             stmt2.execute();              x=x+1;             if(x % 1000==0) system.out.println("-----------"+x);

a profiling suggest 90% time spend on getconnection , 10% spend on execute. on single machine, insert speed around 1000 record/s without commit, think very slow. how improve upsert speed in case?


-

Comments

Post a Comment

Popular posts from this blog

qt - QML MouseArea onWheel event not working properly when inside QML Scrollview -

i have mousearea inside scrollview inside rectangle . implemented zoom feature zooms in/out when pressing ctrl , scrolling mouse wheel. however, zooms in when scrollview way @ top, , zooms out when scrollview way @ bottom. there additional logic handle external drag , drop of files. issue should able replicated long text inside textedit big enough scrollview . apparently bug before, can't work properly. tried solution in following link: qtquick2: handle onwheel event inside of scrollview rectangle { id: palgenrectangle layout.minimumwidth: 50 property string display //width:800 color: "white" scrollview { id: palgentextscrollview anchors.fill: parent mousearea { id: mousearea anchors.fill: parent onwheel: { if (wheel.modifiers & qt.controlmodifier){ if (wheel.angledelta.y > 0) { mainte...
Read more

java - is not an enclosing class / new Intent Cannot Resolve Constructor -

thanks checking out question! i have relativly ismple loop add onclicklistener each imageview add layout, when trying add new intent it, gives me 1 of following errors: string[] imageurls = imageurlsstring.split("/"); (int = 0; < imageurls.length; i++){ imageview image = new imageview(this); image.setlayoutparams(new android.view.viewgroup.layoutparams(getpx(182),getpx(256))); image.setpadding(getpx(3),getpx(3),getpx(3),getpx(3)); final string imageurl = ".../images/" + imageurls[i]; picasso.with(this).load(imageurl).into(image); image.setonclicklistener(new view.onclicklistener() { @override public void onclick(view view) { intent intent; intent = new intent(this, imageactivity.class); intent.putextra("url", imageurl); startactivity(intent); } }); linearlayout.addview(image)...
Read more

python - Error importing VideoFileClip from moviepy : AttributeError: 'PermissionError' object has no attribute 'message' -

i'm using jupyter notebook. have tried anaconda console well. tried importing both ways shown below from moviepy.editor import videofileclip moviepy.video.io.videofileclip import videofileclip both of them gave me same error. full trace below attributeerror traceback (most recent call last) <ipython-input-10-9afa9d6e87c4> in <module>() 6 import glob 7 import math ----> 8 moviepy.editor import videofileclip 9 moviepy.video.io.videofileclip import videofileclip c:\program files\anaconda3\lib\site-packages\moviepy\editor.py in <module>() 20 # clips 21 ---> 22 .video.io.videofileclip import videofileclip 23 .video.io.imagesequenceclip import imagesequenceclip 24 .video.io.downloader import download_webfile c:\program files\anaconda3\lib\site-packages\moviepy\video\io\videofileclip.py in <module>() 1 import os 2 ----> 3 moviepy.video.videoclip import videoclip ...
Read more