apache spark - How to get size (metadata) of a message without fetch request in Kafka through SimpleConsumer? -

-

i using simpleconsumer , trying pull message size wise (bytes) using spark.

i able earliest , latest offset using metadata request, dont know how number of bytes in kafka (0.8.0).

i dont want use fetch request because want check sufficient data run spark job (not streaming) perform action.

from metadata response cant size, size need download message meaningless.


Comments

Post a Comment

Popular posts from this blog

rewrite asp to php based on asp query -

-
i have seen several questions similar , tried solutions @ loss. need redirect example.com/products.asp?lob=health to example.com/services/healthcare/ i need several lob= different value . there way can redirect each url. maybe kind of rewrite condition on lob if value of lob = health, redirect here. if equals home, redirect somewhere else? not sure "correct" way works. rewritecond %{query_string} ^lob=value1$ [nc] rewriterule ^products.asp$ http://example.org/sample/one/? [r=301,l] and different value: rewritecond %{query_string} ^lob=value2$ [nc] rewriterule ^products.asp$ http://example.org/sample/two/? [r=301,l] and doesn't match value: rewriterule ^products\.asp$ [insert new url here] [r=301,l] the last example works if want redirect .asp url .php. keep in mind have other conditions , rules removing .php extension , adding / files: rewritecond %{request_filename} !-f rewriterule ^([^/]+)/$ $1.php rewriterule ^([^/]+)/([^/]+)/$ /$1...
Read more

android - questions about switching from C2DM to GCM -

-
i'm converting app c2dm gcm , have questions. i've modified gcmquickstart example found here: git clone https://github.com/googlesamples/google-services.git in class extends application, i've got: public class myapp extends application { @override public void oncreate() { super.oncreate(); if (this.checkplayservices()) { intent intent = new intent(this, registrationintentservice.class); startservice(intent); } else{ log.e(tag,"failed checkforgcm"); } } } afaik, starts service , it'll run forever or if operating system shuts down whereupon operating system restart it. correct? checkplayservices() makes sure android 2.3 or greater , google play services installed these requirements use gcm. correct? public class registrationintentservice extends intentservice{ private static final string tag = "regintentservice"; public registratio...
Read more

visual studio 2010 - Error on RDLC Expression -

-
i using exp calculate total fees paid on rdlc report: =sum(iif(fields!responsedescription.value ="approved successful",int(fields!amount.value), 0)) and #error in resulting column , can issue ? . and similar exp above work fine : =sum(iif(fields!responsedescription.value <> "",int(fields!amount.value), 0)) few notes: 1- amount integer , present. 2- responsedescription string , present. thank you you can use expression: =sum(cint(iif(fields!responsedescription.value ="approved successful", fields!amount.value, 0))) you have convert every possible values same type before aggregation. i think second expression works fine because in true case ( fields!responsedescription.value <> "" ) used expression converted integer .
Read more