performance - apache phoenix, slow write speed when change currentSCN -


have old hbase table (billion records) , need use data create new phoenix table. want preserve record modtime old record. based on phoenix suggestion. give property currentscn when connect. seems slow. while doc

keep in mind creating new connection not expensive operation. same underlying hconnection used connections same cluster, it’s more or less instantiating few objects.

i test using simple program.

        class.forname("org.apache.phoenix.jdbc.phoenixdriver");         int x = 0;         for(int i=1000000 ;i< 2000000;i++ ) {             properties prop = new properties();             prop.setproperty("currentscn",string.valueof(i));             connection con = drivermanager.getconnection("jdbc:phoenix:a1.cluster",prop);             con.setautocommit(false);              preparedstatement  stmt2 = con.preparestatement("upsert fff values (?,?,?,?,?)");             stmt2.setstring(1,string.valueof(i));             stmt2.settimestamp(2, timestamp.from(instant.ofepochmilli(system.currenttimemillis())));             stmt2.setstring(3, null);             stmt2.setdouble(4, double.valueof(i));             stmt2.setstring(5, null);             stmt2.execute();              x=x+1;             if(x % 1000==0) system.out.println("-----------"+x);

a profiling suggest 90% time spend on getconnection , 10% spend on execute. on single machine, insert speed around 1000 record/s without commit, think very slow. how improve upsert speed in case?


-

Comments

Post a Comment

Popular posts from this blog

python - How to insert QWidgets in the middle of a Layout? -

Image
i'm using qt framework build graphical user interface. use qgridlayout position qwidgets neatly. gui looks this: my application regularly adds new widgets gui @ runtime. these new widgets not added @ end of qlayout, somewhere in middle. the procedure bit cumbersome. applied on figure above, need take out widg_c , widg_d , ... qgridlayout. next, add widg_x and widg_y , , put other widgets again. how remove widgets qgridlayout: for in reversed(range(mygridlayout.count())): self.itemat(i).widget().setparent(none) ### as long you're dealing small amount of widgets, procedure not disaster. in application display lot of small widgets - perhaps 50 or more! application freezes second while procedure ongoing, annoying user. there way insert widgets somewhere in qlayout, without need take out other widgets? edit: apparently solution qvboxlayout simple. use function insertwidget(..) instead of addwidget(..) . docs can found link: http://doc.qt.io/qt-5/qboxlayou
Read more

python - serve multiple gunicorn django instances under nginx ubuntu -

i need serve 2 webs apps (django) on gunicorn under different domain names using nginx on ubuntu. started using tutorial: https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04 it worked fine 1 using that. tried same thing second gives me 502 gateway second domain. tried following tutorial: http://michal.karzynski.pl/blog/2013/10/29/serving-multiple-django-applications-with-nginx-gunicorn-supervisor/ i'm @ part run gunicorn start script: sudo bin/gunicorn_start i response: starting webuildblack root traceback (most recent call last): file "/home/devin/webuildblack/bin/gunicorn", line 7, in <module> gunicorn.app.wsgiapp import run importerror: no module named 'gunicorn.app'; 'gunicorn' not package you have named program gunicorn.py gunicorn -package. rename file , remove gunicorn.pyc file.
Read more

module - Prestashop displayPaymentReturn hook url -

i new prestashop developer , trying create paymentmodule. have got show payment method can not proceed purchase because not know hot works. does know should redirect run hooddisplaypaymentreturn method? i happy if explain me complete navigation map make purchase. anyway, can find relation between hooks , pages? when have new module payment rely on simplest provided prestashop: bankwire. inside can find 3 hooks. hookpayment: public function hookpayment($params) { if (!$this->active) return; if (!$this->checkcurrency($params['cart'])) return; $this->smarty->assign(array( 'this_path' => $this->_path, 'this_path_bw' => $this->_path, 'this_path_ssl' => tools::getshopdomainssl(true, true).__ps_base_uri__.'modules/'.$this->name.'/' )); return $this->display(__file__, 'payment.tpl'); } hookdisplaypaymenteu: public fun
Read more