apache spark - How to get size (metadata) of a message without fetch request in Kafka through SimpleConsumer? -

-

i using simpleconsumer , trying pull message size wise (bytes) using spark.

i able earliest , latest offset using metadata request, dont know how number of bytes in kafka (0.8.0).

i dont want use fetch request because want check sufficient data run spark job (not streaming) perform action.

from metadata response cant size, size need download message meaningless.


Comments

Post a Comment

Popular posts from this blog

android - questions about switching from C2DM to GCM -

-
i'm converting app c2dm gcm , have questions. i've modified gcmquickstart example found here: git clone https://github.com/googlesamples/google-services.git in class extends application, i've got: public class myapp extends application { @override public void oncreate() { super.oncreate(); if (this.checkplayservices()) { intent intent = new intent(this, registrationintentservice.class); startservice(intent); } else{ log.e(tag,"failed checkforgcm"); } } } afaik, starts service , it'll run forever or if operating system shuts down whereupon operating system restart it. correct? checkplayservices() makes sure android 2.3 or greater , google play services installed these requirements use gcm. correct? public class registrationintentservice extends intentservice{ private static final string tag = "regintentservice"; public registratio...
Read more

c++ - Qt setGeometry: Unable to set geometry -

-
why? setgeometry: unable set geometry 22x22+320+145 on qwidgetwindow/'widgetclasswindow'. resulting geometry: 116x22+320+145 (frame: 8, 30, 8, 8, custom margin: 0, 0, 0, 0, minimum size: 22x22, maximum size: 16777215x16777215). the project is: project.pro qt += core gui greaterthan(qt_major_version, 4): qt += widgets target = untitled5 template = app sources += main.cpp\ widget.cpp headers += widget.h widget.h #ifndef widget_h #define widget_h #include <qwidget> class widget : public qwidget { q_object public: explicit widget(qwidget *parent = 0); ~widget(); private: }; #endif // widget_h widget.cpp #include "widget.h" #include <qvboxlayout> widget::widget(qwidget *parent) : qwidget(parent) { qvboxlayout *vlayout = new qvboxlayout(this); } widget::~widget() { } main.cpp #include "widget.h" #include <qapplication> int main(int argc, char *argv[]) { qapplication a...
Read more

batch file - How to extract all multi-volume RAR archives from subfolders of a folder? -

-
i search way unpack multi-volume archives after download via batch. i download folders .r?? files in on ftp monitoring program , want winrar goes in first subfolder in source folder , start unpacking .r00, delete archive , move folder unpacked files new location. then batch script should start process again next subfolder. let's source folder c:\users\unpack contains following subfolders files: source folder subfolder1 archive1.r00 archive1.r01 archive1.r02 xxx.txt subfolder2 archive2.r00 archive2.r01 yyy.txt subfolder3 archive3.r00 archive3.r01 archive3.r02 archive3.r04 archive3.r05 zzz.txt i had start script in link below, script can't want, have started new question. how unpack rar archives in subfolders of folder , delete archives? the script in link above unrars files in subfolders , moves folder files new location. want script unrars , moves subfolder subfolder in source folder. edit.1 if winrar ready first subfolder s...
Read more