apache spark - How to get size (metadata) of a message without fetch request in Kafka through SimpleConsumer? -

-

i using simpleconsumer , trying pull message size wise (bytes) using spark.

i able earliest , latest offset using metadata request, dont know how number of bytes in kafka (0.8.0).

i dont want use fetch request because want check sufficient data run spark job (not streaming) perform action.

from metadata response cant size, size need download message meaningless.


Comments

Post a Comment

Popular posts from this blog

c++ - Qt setGeometry: Unable to set geometry -

-
why? setgeometry: unable set geometry 22x22+320+145 on qwidgetwindow/'widgetclasswindow'. resulting geometry: 116x22+320+145 (frame: 8, 30, 8, 8, custom margin: 0, 0, 0, 0, minimum size: 22x22, maximum size: 16777215x16777215). the project is: project.pro qt += core gui greaterthan(qt_major_version, 4): qt += widgets target = untitled5 template = app sources += main.cpp\ widget.cpp headers += widget.h widget.h #ifndef widget_h #define widget_h #include <qwidget> class widget : public qwidget { q_object public: explicit widget(qwidget *parent = 0); ~widget(); private: }; #endif // widget_h widget.cpp #include "widget.h" #include <qvboxlayout> widget::widget(qwidget *parent) : qwidget(parent) { qvboxlayout *vlayout = new qvboxlayout(this); } widget::~widget() { } main.cpp #include "widget.h" #include <qapplication> int main(int argc, char *argv[]) { qapplication a...
Read more

python - argument must be rect style object - Pygame -

-
disclaimer: know other people on site have had error it's been in different contexts , fixes don't seem work here. hey stackoverflow, i've been making sideways space invaders game, ship follows mouse's y position stays on left, , clicking shoots bullet. enemies spawn randomly right , fly left, , objects removed once each side of screen. works fine, when tried implement collision detection see when shot hits enemy, error: typeerror: argument must rect style object. it points line "return hitbox.colliderect(target.rect)". try , work out causing made 'print' lines below see 'target.rect' (and whether proper rect object), , seemed same style shot's own rect, i'm lost why colliderect doesn't accept it. i've annotated pretty everything, sorry if it's insult intelligence should make things easier. class causes problems: class gameobject: def __init__(self, image, height, speed): self.speed = speed self.image...
Read more

How to resolve Delphi error: Incompatible types: 'PWideChar' and 'Pointer' -

-
i have piece of code delphi 7: var lprgndata: prgndata; pc: pchar; pr: prect; ... pc := @(lprgndata^.buffer[0]); in delphi xe4 gives following compile error: incompatible types: 'pwidechar' , 'pointer' how should code updated work correctly in xe4? thanks whether or not compiles depends upon setting of type-checked pointers option . have enabled option excellent decision. doing results in stricter type checking. with type-checked pointers disabled, code compile. type-checked pointers enabled, code not compile, want because code not valid. now, on types in question. defined in windows unit this: type prgndata = ^trgndata; {$externalsym _rgndata} _rgndata = record rdh: trgndataheader; buffer: array[0..0] of byte; reserved: array[0..2] of byte; end; trgndata = _rgndata; {$externalsym rgndata} rgndata = _rgndata; the benefit of using type-checked pointers compiler can tell doing not valid. knows lprgndat...
Read more