| Message ID | 1469497794-16976-2-git-send-email-zhangchen.fnst@cn.fujitsu.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 2016年07月26日 09:49, Zhang Chen wrote: > This a COLO net ascii figure: > > Primary qemu Secondary qemu > +----------------------------------------------------------+ +----------------------------------------------------------------+ > | +-----------------------------------------------------+ | | +-----------------------------------------------------------+ | > | | | | | | | | > | | guest | | | | guest | | > | | | | | | | | > | +-------^------------------+--------------------------+ | | +---------------------+--------+----------------------------+ | > | | | | | ^ | | > | | | | | | | | > | | +------------------------------------------------------+ | | | | > |netfilter| | | | | | netfilter | | | > | +----------+ -----------------------+ | | | +-----------------------------------------------------------+ | > | | | | | | | | | | | | filter excute order | | > | | | | | | | | | | | | +-------------------> | | > | | | | | | | | | | | | TCP | | > | | +-----+--+--+ +------v-----+ | +------------+ | | | | +------------+ +---+----+---v+rewriter++ +------------+ | | > | | | | | | | | | | | | | | | | | | | | | | > | | | filter | | filter +----> colo <--------+ +--------> filter +--> adjust | adjust +--> filter | | | > | | | mirror | | redirector | | | compare | | | | | | redirector | | ack | seq | | redirector | | | > | | | | | | | | | | | | | | | | | | | | | | > | | +----^------+ +------------+ | +-----+------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | > | | | tx rx | | | | | | tx all | rx | | > | | | | | | | | +-----------------------------------------------------------+ | > | | | | | | | | | | > | | | filter excute order | | | | | | | > | | | +-------------------> | | | +------------------------------------------------------------+ | > | +-----------------------------------+ | | | | > | | | | | | > +----------------------------------------------------------+ +----------------------------------------------------------------+ > |guest receive |guest send > | | > +--------+------------------------------------v------------+ > | | NOTE: filter direction is rx/tx/all > | tap | rx:receive packets sent to the netdev > | | tx:receive packets sent by the netdev > +----------------------------------------------------------+ > > In COLO-compare. > Packets coming from the primary char indev will be sent to outdev > Packets coming from the secondary char dev will be dropped > colo-comapre need two input chardev and one output chardev: > primary_in=chardev1-id > secondary_in=chardev2-id > outdev=chardev3-id Though it has 'compare' in its name, this description needs some key information still. e.g what it did (packet comparing). And then you can describe where were the data sources from. > usage: > > primary: > -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown > -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 > -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait > -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait > -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait > -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 > -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait > -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 > -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 > -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out Looks like this filter were missed in the above figure. > -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 > -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 > > secondary: > -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown > -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 > -chardev socket,id=red0,host=3.3.3.3,port=9003 > -chardev socket,id=red1,host=3.3.3.3,port=9004 > -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 > -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 > > Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> > Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> > Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> > --- > net/Makefile.objs | 1 + > net/colo-compare.c | 222 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > qemu-options.hx | 38 +++++++++ > vl.c | 3 +- > 4 files changed, 263 insertions(+), 1 deletion(-) > create mode 100644 net/colo-compare.c > > diff --git a/net/Makefile.objs b/net/Makefile.objs > index b7c22fd..ba92f73 100644 > --- a/net/Makefile.objs > +++ b/net/Makefile.objs > @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o > common-obj-y += filter.o > common-obj-y += filter-buffer.o > common-obj-y += filter-mirror.o > +common-obj-y += colo-compare.o > diff --git a/net/colo-compare.c b/net/colo-compare.c > new file mode 100644 > index 0000000..0402958 > --- /dev/null > +++ b/net/colo-compare.c > @@ -0,0 +1,222 @@ > +/* > + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) > + * (a.k.a. Fault Tolerance or Continuous Replication) > + * > + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. > + * Copyright (c) 2016 FUJITSU LIMITED > + * Copyright (c) 2016 Intel Corporation > + * > + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> > + * > + * This work is licensed under the terms of the GNU GPL, version 2 or > + * later. See the COPYING file in the top-level directory. > + */ > + > +#include "qemu/osdep.h" > +#include "qemu/error-report.h" > +#include "qemu-common.h" > +#include "qapi/qmp/qerror.h" > +#include "qapi/error.h" > +#include "net/net.h" > +#include "net/vhost_net.h" > +#include "qom/object_interfaces.h" > +#include "qemu/iov.h" > +#include "qom/object.h" > +#include "qemu/typedefs.h" > +#include "net/queue.h" > +#include "sysemu/char.h" > +#include "qemu/sockets.h" > +#include "qapi-visit.h" > + > +#define TYPE_COLO_COMPARE "colo-compare" > +#define COLO_COMPARE(obj) \ > + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) > + > +#define COMPARE_READ_LEN_MAX NET_BUFSIZE > + > +typedef struct CompareState { > + Object parent; > + > + char *pri_indev; > + char *sec_indev; > + char *outdev; > + CharDriverState *chr_pri_in; > + CharDriverState *chr_sec_in; > + CharDriverState *chr_out; > + QTAILQ_ENTRY(CompareState) next; > + SocketReadState pri_rs; > + SocketReadState sec_rs; > +} CompareState; > + > +typedef struct CompareClass { > + ObjectClass parent_class; > +} CompareClass; > + > +static char *compare_get_pri_indev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->pri_indev); > +} > + > +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->pri_indev); > + s->pri_indev = g_strdup(value); > +} > + > +static char *compare_get_sec_indev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->sec_indev); > +} > + > +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->sec_indev); > + s->sec_indev = g_strdup(value); > +} > + > +static char *compare_get_outdev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->outdev); > +} > + > +static void compare_set_outdev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->outdev); > + s->outdev = g_strdup(value); > +} > + > +static void compare_pri_rs_finalize(SocketReadState *pri_rs) > +{ > + /* if packet_enqueue pri pkt failed we will send unsupported packet */ > +} > + > +static void compare_sec_rs_finalize(SocketReadState *sec_rs) > +{ > + /* if packet_enqueue sec pkt failed we will notify trace */ > +} > + > +/* > + * called from the main thread on the primary > + * to setup colo-compare. > + */ > +static void colo_compare_complete(UserCreatable *uc, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(uc); > + > + if (!s->pri_indev || !s->sec_indev || !s->outdev) { > + error_setg(errp, "colo compare needs 'primary_in' ," > + "'secondary_in','outdev' property set"); > + return; > + } else if (!strcmp(s->pri_indev, s->outdev) || > + !strcmp(s->sec_indev, s->outdev) || > + !strcmp(s->pri_indev, s->sec_indev)) { > + error_setg(errp, "'indev' and 'outdev' could not be same " > + "for compare module"); > + return; Can relax this by reusing the socket chardev? Looks we only use one direction during packet passing. > + } > + > + s->chr_pri_in = qemu_chr_find(s->pri_indev); > + if (s->chr_pri_in == NULL) { > + error_setg(errp, "Primary IN Device '%s' not found", > + s->pri_indev); > + return; > + } > + > + s->chr_sec_in = qemu_chr_find(s->sec_indev); > + if (s->chr_sec_in == NULL) { > + error_setg(errp, "Secondary IN Device '%s' not found", > + s->sec_indev); > + return; > + } > + > + s->chr_out = qemu_chr_find(s->outdev); > + if (s->chr_out == NULL) { > + error_setg(errp, "OUT Device '%s' not found", s->outdev); > + return; > + } Like I point out in the past iterations, we probably need some inspection here and fail if it was socket chardev. (Similar to what vhost-use did for udp socket). > + > + qemu_chr_fe_claim_no_fail(s->chr_pri_in); > + > + qemu_chr_fe_claim_no_fail(s->chr_sec_in); > + > + qemu_chr_fe_claim_no_fail(s->chr_out); > + > + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); > + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); > + Why chr_out is left here? > + return; > +} > + > +static void colo_compare_class_init(ObjectClass *oc, void *data) > +{ > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > + > + ucc->complete = colo_compare_complete; > +} > + > +static void colo_compare_init(Object *obj) > +{ > + object_property_add_str(obj, "primary_in", > + compare_get_pri_indev, compare_set_pri_indev, > + NULL); > + object_property_add_str(obj, "secondary_in", > + compare_get_sec_indev, compare_set_sec_indev, > + NULL); > + object_property_add_str(obj, "outdev", > + compare_get_outdev, compare_set_outdev, > + NULL); > +} > + > +static void colo_compare_finalize(Object *obj) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + if (s->chr_pri_in) { > + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); > + qemu_chr_fe_release(s->chr_pri_in); > + } > + if (s->chr_sec_in) { > + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); > + qemu_chr_fe_release(s->chr_sec_in); > + } > + if (s->chr_out) { > + qemu_chr_fe_release(s->chr_out); > + } > + > + g_free(s->pri_indev); > + g_free(s->sec_indev); > + g_free(s->outdev); > +} > + > +static const TypeInfo colo_compare_info = { > + .name = TYPE_COLO_COMPARE, > + .parent = TYPE_OBJECT, > + .instance_size = sizeof(CompareState), > + .instance_init = colo_compare_init, > + .instance_finalize = colo_compare_finalize, > + .class_size = sizeof(CompareClass), > + .class_init = colo_compare_class_init, > + .interfaces = (InterfaceInfo[]) { > + { TYPE_USER_CREATABLE }, > + { } > + } > +}; > + > +static void register_types(void) > +{ > + type_register_static(&colo_compare_info); > +} > + > +type_init(register_types); > diff --git a/qemu-options.hx b/qemu-options.hx > index 587de8f..79e5896 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -3866,6 +3866,44 @@ Dump the network traffic on netdev @var{dev} to the file specified by > The file format is libpcap, so it can be analyzed with tools such as tcpdump > or Wireshark. > > +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, > +outdev=@var{chardevid} > + > +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid},than compare primary packet with > +secondary packet. if the packet same,we will output primary s/if/If/ and miss a blank before ',' > +packet to outdev@var{chardevid},else we will notify colo-frame miss a blank before ',' > +do checkpoint and send primary packet to outdev@var{chardevid}. > + > +we can use it with the help of filter-mirror and filter-redirector. > + > +The simple usage: This line could be removed since we have example below. > + > +@example > + > +primary: > +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown > +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 > +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait > +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait > +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait > +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 > +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait > +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 > +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 > +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out > +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 > +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 > + > +secondary: > +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown > +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 > +-chardev socket,id=red0,host=3.3.3.3,port=9003 > +-chardev socket,id=red1,host=3.3.3.3,port=9004 > +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 > +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 > + > +@end example > + Better add detail description on the above command line since it was rather complex for starters. > @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] > @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] > > diff --git a/vl.c b/vl.c > index cbe51ac..c6b9a6f 100644 > --- a/vl.c > +++ b/vl.c > @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) > if (g_str_equal(type, "filter-buffer") || > g_str_equal(type, "filter-dump") || > g_str_equal(type, "filter-mirror") || > - g_str_equal(type, "filter-redirector")) { > + g_str_equal(type, "filter-redirector") || > + g_str_equal(type, "colo-compare")) { > return false; > } >
On 08/02/2016 02:26 PM, Jason Wang wrote: > > > On 2016年07月26日 09:49, Zhang Chen wrote: >> This a COLO net ascii figure: >> >> Primary qemu Secondary qemu >> +----------------------------------------------------------+ >> +----------------------------------------------------------------+ >> | +-----------------------------------------------------+ | >> | +-----------------------------------------------------------+ | >> | | | | >> | | | | >> | | guest | | >> | | guest | | >> | | | | >> | | | | >> | +-------^------------------+--------------------------+ | >> | +---------------------+--------+----------------------------+ | >> | | | | | ^ >> | | >> | | | | | | >> | | >> | | +------------------------------------------------------+ >> | | | | >> |netfilter| | | | | | netfilter | >> | | >> | +----------+ -----------------------+ | | | >> +-----------------------------------------------------------+ | >> | | | | | | | | | >> | | | filter excute order | | >> | | | | | | | | | >> | | | +-------------------> | | >> | | | | | | | | | >> | | | TCP | | >> | | +-----+--+--+ +------v-----+ | +------------+ | | | >> | +------------+ +---+----+---v+rewriter++ +------------+ | | >> | | | | | | | | | | | | >> | | | | | | | | | | >> | | | filter | | filter +----> colo <--------+ >> +--------> filter +--> adjust | adjust +--> filter | | | >> | | | mirror | | redirector | | | compare | | | >> | | | redirector | | ack | seq | | redirector | | | >> | | | | | | | | | | | >> | | | | | | | | | | | >> | | +----^------+ +------------+ | +-----+------+ | | >> | | +------------+ +--------+--------------+ +---+--------+ | | >> | | | tx rx | | | | >> | | tx all | rx | | >> | | | | | | | >> | +-----------------------------------------------------------+ | >> | | | | | | | >> | | | >> | | | filter excute order | | | | >> | | | >> | | | +-------------------> | | | >> +------------------------------------------------------------+ | >> | +-----------------------------------+ | | | | >> | | | | | | >> +----------------------------------------------------------+ >> +----------------------------------------------------------------+ >> |guest receive |guest send >> | | >> +--------+------------------------------------v------------+ >> | | NOTE: filter direction is rx/tx/all >> | tap | >> rx:receive packets sent to the netdev >> | | tx:receive packets sent by the netdev >> +----------------------------------------------------------+ >> >> In COLO-compare. >> Packets coming from the primary char indev will be sent to outdev >> Packets coming from the secondary char dev will be dropped >> colo-comapre need two input chardev and one output chardev: >> primary_in=chardev1-id >> secondary_in=chardev2-id >> outdev=chardev3-id > > Though it has 'compare' in its name, this description needs some key > information still. e.g what it did (packet comparing). And then you > can describe where were the data sources from. OK, I will add some comments for this. > >> usage: >> >> primary: >> -netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> -object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out > > Looks like this filter were missed in the above figure. > Yes,I will update figure in next version. >> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> -object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> >> secondary: >> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=red0,host=3.3.3.3,port=9003 >> -chardev socket,id=red1,host=3.3.3.3,port=9004 >> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> >> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >> --- >> net/Makefile.objs | 1 + >> net/colo-compare.c | 222 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> qemu-options.hx | 38 +++++++++ >> vl.c | 3 +- >> 4 files changed, 263 insertions(+), 1 deletion(-) >> create mode 100644 net/colo-compare.c >> >> diff --git a/net/Makefile.objs b/net/Makefile.objs >> index b7c22fd..ba92f73 100644 >> --- a/net/Makefile.objs >> +++ b/net/Makefile.objs >> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >> common-obj-y += filter.o >> common-obj-y += filter-buffer.o >> common-obj-y += filter-mirror.o >> +common-obj-y += colo-compare.o >> diff --git a/net/colo-compare.c b/net/colo-compare.c >> new file mode 100644 >> index 0000000..0402958 >> --- /dev/null >> +++ b/net/colo-compare.c >> @@ -0,0 +1,222 @@ >> +/* >> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service >> (COLO) >> + * (a.k.a. Fault Tolerance or Continuous Replication) >> + * >> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >> + * Copyright (c) 2016 FUJITSU LIMITED >> + * Copyright (c) 2016 Intel Corporation >> + * >> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> + * >> + * This work is licensed under the terms of the GNU GPL, version 2 or >> + * later. See the COPYING file in the top-level directory. >> + */ >> + >> +#include "qemu/osdep.h" >> +#include "qemu/error-report.h" >> +#include "qemu-common.h" >> +#include "qapi/qmp/qerror.h" >> +#include "qapi/error.h" >> +#include "net/net.h" >> +#include "net/vhost_net.h" >> +#include "qom/object_interfaces.h" >> +#include "qemu/iov.h" >> +#include "qom/object.h" >> +#include "qemu/typedefs.h" >> +#include "net/queue.h" >> +#include "sysemu/char.h" >> +#include "qemu/sockets.h" >> +#include "qapi-visit.h" >> + >> +#define TYPE_COLO_COMPARE "colo-compare" >> +#define COLO_COMPARE(obj) \ >> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >> + >> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >> + >> +typedef struct CompareState { >> + Object parent; >> + >> + char *pri_indev; >> + char *sec_indev; >> + char *outdev; >> + CharDriverState *chr_pri_in; >> + CharDriverState *chr_sec_in; >> + CharDriverState *chr_out; >> + QTAILQ_ENTRY(CompareState) next; >> + SocketReadState pri_rs; >> + SocketReadState sec_rs; >> +} CompareState; >> + >> +typedef struct CompareClass { >> + ObjectClass parent_class; >> +} CompareClass; >> + >> +static char *compare_get_pri_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->pri_indev); >> +} >> + >> +static void compare_set_pri_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->pri_indev); >> + s->pri_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_sec_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->sec_indev); >> +} >> + >> +static void compare_set_sec_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->sec_indev); >> + s->sec_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_outdev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->outdev); >> +} >> + >> +static void compare_set_outdev(Object *obj, const char *value, Error >> **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->outdev); >> + s->outdev = g_strdup(value); >> +} >> + >> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >> +{ >> + /* if packet_enqueue pri pkt failed we will send unsupported >> packet */ >> +} >> + >> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >> +{ >> + /* if packet_enqueue sec pkt failed we will notify trace */ >> +} >> + >> +/* >> + * called from the main thread on the primary >> + * to setup colo-compare. >> + */ >> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(uc); >> + >> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >> + error_setg(errp, "colo compare needs 'primary_in' ," >> + "'secondary_in','outdev' property set"); >> + return; >> + } else if (!strcmp(s->pri_indev, s->outdev) || >> + !strcmp(s->sec_indev, s->outdev) || >> + !strcmp(s->pri_indev, s->sec_indev)) { >> + error_setg(errp, "'indev' and 'outdev' could not be same " >> + "for compare module"); >> + return; > > Can relax this by reusing the socket chardev? Looks we only use one > direction during packet passing. We can't qemu_chr_fe_claim_no_fail(xxx) same one chardev twice. Do you mean we should use -chardev socket,id=xx,....mux=on,... ? > >> + } >> + >> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >> + if (s->chr_pri_in == NULL) { >> + error_setg(errp, "Primary IN Device '%s' not found", >> + s->pri_indev); >> + return; >> + } >> + >> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >> + if (s->chr_sec_in == NULL) { >> + error_setg(errp, "Secondary IN Device '%s' not found", >> + s->sec_indev); >> + return; >> + } >> + >> + s->chr_out = qemu_chr_find(s->outdev); >> + if (s->chr_out == NULL) { >> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >> + return; >> + } > > Like I point out in the past iterations, we probably need some > inspection here and fail if it was socket chardev. (Similar to what > vhost-use did for udp socket). > OK,I got it. >> + >> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_out); >> + >> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >> + > > Why chr_out is left here? Because chr_out needn't read data from chardev, just output. > >> + return; >> +} >> + >> +static void colo_compare_class_init(ObjectClass *oc, void *data) >> +{ >> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >> + >> + ucc->complete = colo_compare_complete; >> +} >> + >> +static void colo_compare_init(Object *obj) >> +{ >> + object_property_add_str(obj, "primary_in", >> + compare_get_pri_indev, >> compare_set_pri_indev, >> + NULL); >> + object_property_add_str(obj, "secondary_in", >> + compare_get_sec_indev, >> compare_set_sec_indev, >> + NULL); >> + object_property_add_str(obj, "outdev", >> + compare_get_outdev, compare_set_outdev, >> + NULL); >> +} >> + >> +static void colo_compare_finalize(Object *obj) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + if (s->chr_pri_in) { >> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_pri_in); >> + } >> + if (s->chr_sec_in) { >> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_sec_in); >> + } >> + if (s->chr_out) { >> + qemu_chr_fe_release(s->chr_out); >> + } >> + >> + g_free(s->pri_indev); >> + g_free(s->sec_indev); >> + g_free(s->outdev); >> +} >> + >> +static const TypeInfo colo_compare_info = { >> + .name = TYPE_COLO_COMPARE, >> + .parent = TYPE_OBJECT, >> + .instance_size = sizeof(CompareState), >> + .instance_init = colo_compare_init, >> + .instance_finalize = colo_compare_finalize, >> + .class_size = sizeof(CompareClass), >> + .class_init = colo_compare_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { TYPE_USER_CREATABLE }, >> + { } >> + } >> +}; >> + >> +static void register_types(void) >> +{ >> + type_register_static(&colo_compare_info); >> +} >> + >> +type_init(register_types); >> diff --git a/qemu-options.hx b/qemu-options.hx >> index 587de8f..79e5896 100644 >> --- a/qemu-options.hx >> +++ b/qemu-options.hx >> @@ -3866,6 +3866,44 @@ Dump the network traffic on netdev @var{dev} >> to the file specified by >> The file format is libpcap, so it can be analyzed with tools such >> as tcpdump >> or Wireshark. >> +@item -object >> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >> +outdev=@var{chardevid} >> + >> +Colo-compare gets packet from primary_in@var{chardevid} and >> secondary_in@var{chardevid},than compare primary packet with >> +secondary packet. if the packet same,we will output primary > > s/if/If/ and miss a blank before ',' will fix~ >> +packet to outdev@var{chardevid},else we will notify colo-frame > > miss a blank before ',' will fix it. > >> +do checkpoint and send primary packet to outdev@var{chardevid}. >> + >> +we can use it with the help of filter-mirror and filter-redirector. >> + >> +The simple usage: > > This line could be removed since we have example below. will remove it. > >> + >> +@example >> + >> +primary: >> +-netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> +-object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> +-object >> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> +-object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> + >> +secondary: >> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> + >> +@end example >> + > > Better add detail description on the above command line since it was > rather complex for starters. We have add detail description in commit log, need we repeat it again here? Thanks Zhang Chen > > >> @item -object >> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> @item -object >> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> diff --git a/vl.c b/vl.c >> index cbe51ac..c6b9a6f 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >> *type) >> if (g_str_equal(type, "filter-buffer") || >> g_str_equal(type, "filter-dump") || >> g_str_equal(type, "filter-mirror") || >> - g_str_equal(type, "filter-redirector")) { >> + g_str_equal(type, "filter-redirector") || >> + g_str_equal(type, "colo-compare")) { >> return false; >> } > > > > . >
diff --git a/net/Makefile.objs b/net/Makefile.objs index b7c22fd..ba92f73 100644 --- a/net/Makefile.objs +++ b/net/Makefile.objs @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o common-obj-y += filter.o common-obj-y += filter-buffer.o common-obj-y += filter-mirror.o +common-obj-y += colo-compare.o diff --git a/net/colo-compare.c b/net/colo-compare.c new file mode 100644 index 0000000..0402958 --- /dev/null +++ b/net/colo-compare.c @@ -0,0 +1,222 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. + * Copyright (c) 2016 FUJITSU LIMITED + * Copyright (c) 2016 Intel Corporation + * + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qemu-common.h" +#include "qapi/qmp/qerror.h" +#include "qapi/error.h" +#include "net/net.h" +#include "net/vhost_net.h" +#include "qom/object_interfaces.h" +#include "qemu/iov.h" +#include "qom/object.h" +#include "qemu/typedefs.h" +#include "net/queue.h" +#include "sysemu/char.h" +#include "qemu/sockets.h" +#include "qapi-visit.h" + +#define TYPE_COLO_COMPARE "colo-compare" +#define COLO_COMPARE(obj) \ + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) + +#define COMPARE_READ_LEN_MAX NET_BUFSIZE + +typedef struct CompareState { + Object parent; + + char *pri_indev; + char *sec_indev; + char *outdev; + CharDriverState *chr_pri_in; + CharDriverState *chr_sec_in; + CharDriverState *chr_out; + QTAILQ_ENTRY(CompareState) next; + SocketReadState pri_rs; + SocketReadState sec_rs; +} CompareState; + +typedef struct CompareClass { + ObjectClass parent_class; +} CompareClass; + +static char *compare_get_pri_indev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->pri_indev); +} + +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->pri_indev); + s->pri_indev = g_strdup(value); +} + +static char *compare_get_sec_indev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->sec_indev); +} + +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->sec_indev); + s->sec_indev = g_strdup(value); +} + +static char *compare_get_outdev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->outdev); +} + +static void compare_set_outdev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->outdev); + s->outdev = g_strdup(value); +} + +static void compare_pri_rs_finalize(SocketReadState *pri_rs) +{ + /* if packet_enqueue pri pkt failed we will send unsupported packet */ +} + +static void compare_sec_rs_finalize(SocketReadState *sec_rs) +{ + /* if packet_enqueue sec pkt failed we will notify trace */ +} + +/* + * called from the main thread on the primary + * to setup colo-compare. + */ +static void colo_compare_complete(UserCreatable *uc, Error **errp) +{ + CompareState *s = COLO_COMPARE(uc); + + if (!s->pri_indev || !s->sec_indev || !s->outdev) { + error_setg(errp, "colo compare needs 'primary_in' ," + "'secondary_in','outdev' property set"); + return; + } else if (!strcmp(s->pri_indev, s->outdev) || + !strcmp(s->sec_indev, s->outdev) || + !strcmp(s->pri_indev, s->sec_indev)) { + error_setg(errp, "'indev' and 'outdev' could not be same " + "for compare module"); + return; + } + + s->chr_pri_in = qemu_chr_find(s->pri_indev); + if (s->chr_pri_in == NULL) { + error_setg(errp, "Primary IN Device '%s' not found", + s->pri_indev); + return; + } + + s->chr_sec_in = qemu_chr_find(s->sec_indev); + if (s->chr_sec_in == NULL) { + error_setg(errp, "Secondary IN Device '%s' not found", + s->sec_indev); + return; + } + + s->chr_out = qemu_chr_find(s->outdev); + if (s->chr_out == NULL) { + error_setg(errp, "OUT Device '%s' not found", s->outdev); + return; + } + + qemu_chr_fe_claim_no_fail(s->chr_pri_in); + + qemu_chr_fe_claim_no_fail(s->chr_sec_in); + + qemu_chr_fe_claim_no_fail(s->chr_out); + + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); + + return; +} + +static void colo_compare_class_init(ObjectClass *oc, void *data) +{ + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); + + ucc->complete = colo_compare_complete; +} + +static void colo_compare_init(Object *obj) +{ + object_property_add_str(obj, "primary_in", + compare_get_pri_indev, compare_set_pri_indev, + NULL); + object_property_add_str(obj, "secondary_in", + compare_get_sec_indev, compare_set_sec_indev, + NULL); + object_property_add_str(obj, "outdev", + compare_get_outdev, compare_set_outdev, + NULL); +} + +static void colo_compare_finalize(Object *obj) +{ + CompareState *s = COLO_COMPARE(obj); + + if (s->chr_pri_in) { + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); + qemu_chr_fe_release(s->chr_pri_in); + } + if (s->chr_sec_in) { + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); + qemu_chr_fe_release(s->chr_sec_in); + } + if (s->chr_out) { + qemu_chr_fe_release(s->chr_out); + } + + g_free(s->pri_indev); + g_free(s->sec_indev); + g_free(s->outdev); +} + +static const TypeInfo colo_compare_info = { + .name = TYPE_COLO_COMPARE, + .parent = TYPE_OBJECT, + .instance_size = sizeof(CompareState), + .instance_init = colo_compare_init, + .instance_finalize = colo_compare_finalize, + .class_size = sizeof(CompareClass), + .class_init = colo_compare_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +static void register_types(void) +{ + type_register_static(&colo_compare_info); +} + +type_init(register_types); diff --git a/qemu-options.hx b/qemu-options.hx index 587de8f..79e5896 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -3866,6 +3866,44 @@ Dump the network traffic on netdev @var{dev} to the file specified by The file format is libpcap, so it can be analyzed with tools such as tcpdump or Wireshark. +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, +outdev=@var{chardevid} + +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid},than compare primary packet with +secondary packet. if the packet same,we will output primary +packet to outdev@var{chardevid},else we will notify colo-frame +do checkpoint and send primary packet to outdev@var{chardevid}. + +we can use it with the help of filter-mirror and filter-redirector. + +The simple usage: + +@example + +primary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 + +secondary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=red0,host=3.3.3.3,port=9003 +-chardev socket,id=red1,host=3.3.3.3,port=9004 +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + +@end example + @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] diff --git a/vl.c b/vl.c index cbe51ac..c6b9a6f 100644 --- a/vl.c +++ b/vl.c @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) if (g_str_equal(type, "filter-buffer") || g_str_equal(type, "filter-dump") || g_str_equal(type, "filter-mirror") || - g_str_equal(type, "filter-redirector")) { + g_str_equal(type, "filter-redirector") || + g_str_equal(type, "colo-compare")) { return false; }
This a COLO net ascii figure: Primary qemu Secondary qemu +----------------------------------------------------------+ +----------------------------------------------------------------+ | +-----------------------------------------------------+ | | +-----------------------------------------------------------+ | | | | | | | | | | | guest | | | | guest | | | | | | | | | | | +-------^------------------+--------------------------+ | | +---------------------+--------+----------------------------+ | | | | | | ^ | | | | | | | | | | | | +------------------------------------------------------+ | | | | |netfilter| | | | | | netfilter | | | | +----------+ -----------------------+ | | | +-----------------------------------------------------------+ | | | | | | | | | | | | | filter excute order | | | | | | | | | | | | | | +-------------------> | | | | | | | | | | | | | | TCP | | | | +-----+--+--+ +------v-----+ | +------------+ | | | | +------------+ +---+----+---v+rewriter++ +------------+ | | | | | | | | | | | | | | | | | | | | | | | | | | | filter | | filter +----> colo <--------+ +--------> filter +--> adjust | adjust +--> filter | | | | | | mirror | | redirector | | | compare | | | | | | redirector | | ack | seq | | redirector | | | | | | | | | | | | | | | | | | | | | | | | | | | +----^------+ +------------+ | +-----+------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | | | | tx rx | | | | | | tx all | rx | | | | | | | | | | +-----------------------------------------------------------+ | | | | | | | | | | | | | | filter excute order | | | | | | | | | | +-------------------> | | | +------------------------------------------------------------+ | | +-----------------------------------+ | | | | | | | | | | +----------------------------------------------------------+ +----------------------------------------------------------------+ |guest receive |guest send | | +--------+------------------------------------v------------+ | | NOTE: filter direction is rx/tx/all | tap | rx:receive packets sent to the netdev | | tx:receive packets sent by the netdev +----------------------------------------------------------+ In COLO-compare. Packets coming from the primary char indev will be sent to outdev Packets coming from the secondary char dev will be dropped colo-comapre need two input chardev and one output chardev: primary_in=chardev1-id secondary_in=chardev2-id outdev=chardev3-id usage: primary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 secondary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 -chardev socket,id=red0,host=3.3.3.3,port=9003 -chardev socket,id=red1,host=3.3.3.3,port=9004 -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> --- net/Makefile.objs | 1 + net/colo-compare.c | 222 +++++++++++++++++++++++++++++++++++++++++++++++++++++ qemu-options.hx | 38 +++++++++ vl.c | 3 +- 4 files changed, 263 insertions(+), 1 deletion(-) create mode 100644 net/colo-compare.c