| Message ID | 1471421428-26379-3-git-send-email-zhangchen.fnst@cn.fujitsu.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 2016年08月17日 16:10, Zhang Chen wrote: > This a COLO net ascii figure: > > Primary qemu Secondary qemu > +--------------------------------------------------------------+ +----------------------------------------------------------------+ > | +----------------------------------------------------------+ | | +-----------------------------------------------------------+ | > | | | | | | | | > | | guest | | | | guest | | > | | | | | | | | > | +-------^--------------------------+-----------------------+ | | +---------------------+--------+----------------------------+ | > | | | | | ^ | | > | | | | | | | | > | | +------------------------------------------------------+ | | | | > |netfilter| | | | | | netfilter | | | > | +----------+ +----------------------------+ | | | +-----------------------------------------------------------+ | > | | | | | | out | | | | | | filter excute order | | > | | | | +-----------------------------+ | | | | | | +-------------------> | | > | | | | | | | | | | | | | | TCP | | > | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| | | | +------------+ +---+----+---v+rewriter++ +------------+ | | > | | | | | | | | |in | |in | | | | | | | | | | | | | > | | | filter | | filter | | filter +------> colo <------+ +--------> filter +--> adjust | adjust +--> filter | | | > | | | mirror | |redirector| |redirector| | | compare | | | | | | redirector | | ack | seq | | redirector | | | > | | | | | | | | | | | | | | | | | | | | | | | | > | | +----^-----+ +----+-----+ +----------+ | +---------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | > | | | tx | rx rx | | | | | tx all | rx | | > | | | | | | | | +-----------------------------------------------------------+ | > | | | +--------------+ | | | | | | > | | | filter excute order | | | | | | | > | | | +----------------> | | | +--------------------------------------------------------+ | > | +-----------------------------------------+ | | | > | | | | | | > +--------------------------------------------------------------+ +----------------------------------------------------------------+ > |guest receive | guest send > | | > +--------+----------------------------v------------------------+ > | | NOTE: filter direction is rx/tx/all > | tap | rx:receive packets sent to the netdev > | | tx:receive packets sent by the netdev > +--------------------------------------------------------------+ It's better to add a doc under docs to explain this configuration in detail on top of this series. > In COLO-compare, we do packet comparing job. > Packets coming from the primary char indev will be sent to outdev. > Packets coming from the secondary char dev will be dropped after comparing. > colo-comapre need two input chardev and one output chardev: > primary_in=chardev1-id (source: primary send packet) > secondary_in=chardev2-id (source: secondary send packet) > outdev=chardev3-id > > usage: > > primary: > -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown > -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 > -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait > -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait > -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait > -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 > -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait > -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 > -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 > -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out > -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 > -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 > > secondary: > -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown > -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 > -chardev socket,id=red0,host=3.3.3.3,port=9003 > -chardev socket,id=red1,host=3.3.3.3,port=9004 > -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 > -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 > > Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> > Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> > Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> > --- > net/Makefile.objs | 1 + > net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++ > qemu-options.hx | 39 ++++++++ > vl.c | 3 +- > 4 files changed, 326 insertions(+), 1 deletion(-) > create mode 100644 net/colo-compare.c > > diff --git a/net/Makefile.objs b/net/Makefile.objs > index b7c22fd..ba92f73 100644 > --- a/net/Makefile.objs > +++ b/net/Makefile.objs > @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o > common-obj-y += filter.o > common-obj-y += filter-buffer.o > common-obj-y += filter-mirror.o > +common-obj-y += colo-compare.o > diff --git a/net/colo-compare.c b/net/colo-compare.c > new file mode 100644 > index 0000000..cdc3e0e > --- /dev/null > +++ b/net/colo-compare.c > @@ -0,0 +1,284 @@ > +/* > + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) > + * (a.k.a. Fault Tolerance or Continuous Replication) > + * > + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. > + * Copyright (c) 2016 FUJITSU LIMITED > + * Copyright (c) 2016 Intel Corporation > + * > + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> > + * > + * This work is licensed under the terms of the GNU GPL, version 2 or > + * later. See the COPYING file in the top-level directory. > + */ > + > +#include "qemu/osdep.h" > +#include "qemu/error-report.h" > +#include "qemu-common.h" > +#include "qapi/qmp/qerror.h" > +#include "qapi/error.h" > +#include "net/net.h" > +#include "net/vhost_net.h" Looks unnecessary. > +#include "qom/object_interfaces.h" > +#include "qemu/iov.h" > +#include "qom/object.h" > +#include "qemu/typedefs.h" > +#include "net/queue.h" > +#include "sysemu/char.h" > +#include "qemu/sockets.h" > +#include "qapi-visit.h" > + > +#define TYPE_COLO_COMPARE "colo-compare" > +#define COLO_COMPARE(obj) \ > + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) > + > +#define COMPARE_READ_LEN_MAX NET_BUFSIZE > + > +typedef struct CompareState { > + Object parent; > + > + char *pri_indev; > + char *sec_indev; > + char *outdev; > + CharDriverState *chr_pri_in; > + CharDriverState *chr_sec_in; > + CharDriverState *chr_out; > + QTAILQ_ENTRY(CompareState) next; This looks not used in this series but in commit "colo-compare and filter-rewriter work with colo-frame". We'd better delay the introducing to that patch. > + SocketReadState pri_rs; > + SocketReadState sec_rs; > +} CompareState; > + > +typedef struct CompareClass { > + ObjectClass parent_class; > +} CompareClass; > + > +typedef struct CompareChardevProps { > + bool is_socket; > + bool is_unix; > +} CompareChardevProps; > + > +static char *compare_get_pri_indev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->pri_indev); > +} > + > +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->pri_indev); > + s->pri_indev = g_strdup(value); > +} > + > +static char *compare_get_sec_indev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->sec_indev); > +} > + > +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->sec_indev); > + s->sec_indev = g_strdup(value); > +} > + > +static char *compare_get_outdev(Object *obj, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + return g_strdup(s->outdev); > +} > + > +static void compare_set_outdev(Object *obj, const char *value, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + g_free(s->outdev); > + s->outdev = g_strdup(value); > +} > + > +static void compare_pri_rs_finalize(SocketReadState *pri_rs) > +{ > + /* if packet_enqueue pri pkt failed we will send unsupported packet */ > +} > + > +static void compare_sec_rs_finalize(SocketReadState *sec_rs) > +{ > + /* if packet_enqueue sec pkt failed we will notify trace */ > +} > + > +static int compare_chardev_opts(void *opaque, > + const char *name, const char *value, > + Error **errp) > +{ > + CompareChardevProps *props = opaque; > + > + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { > + props->is_socket = true; > + } else if (strcmp(name, "host") == 0) { Typo? net_vhost_chardev_opts() did: } else if (strcmp(name, "path") == 0) { props->is_unix = true; } > + props->is_unix = true; > + } else if (strcmp(name, "port") == 0) { > + } else if (strcmp(name, "server") == 0) { > + } else if (strcmp(name, "wait") == 0) { > + } else { > + error_setg(errp, > + "COLO-compare does not support a chardev with option %s=%s", > + name, value); > + return -1; > + } > + return 0; > +} > + > +/* > + * called from the main thread on the primary > + * to setup colo-compare. > + */ > +static void colo_compare_complete(UserCreatable *uc, Error **errp) > +{ > + CompareState *s = COLO_COMPARE(uc); > + CompareChardevProps props; > + > + if (!s->pri_indev || !s->sec_indev || !s->outdev) { > + error_setg(errp, "colo compare needs 'primary_in' ," > + "'secondary_in','outdev' property set"); > + return; > + } else if (!strcmp(s->pri_indev, s->outdev) || > + !strcmp(s->sec_indev, s->outdev) || > + !strcmp(s->pri_indev, s->sec_indev)) { > + error_setg(errp, "'indev' and 'outdev' could not be same " > + "for compare module"); > + return; > + } > + > + s->chr_pri_in = qemu_chr_find(s->pri_indev); > + if (s->chr_pri_in == NULL) { > + error_setg(errp, "Primary IN Device '%s' not found", > + s->pri_indev); > + return; > + } > + > + /* inspect chardev opts */ > + memset(&props, 0, sizeof(props)); > + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) { > + return; > + } > + > + if (!props.is_socket || !props.is_unix) { > + error_setg(errp, "chardev \"%s\" is not a unix socket", > + s->pri_indev); > + return; > + } > + > + s->chr_sec_in = qemu_chr_find(s->sec_indev); > + if (s->chr_sec_in == NULL) { > + error_setg(errp, "Secondary IN Device '%s' not found", > + s->sec_indev); > + return; > + } > + > + memset(&props, 0, sizeof(props)); > + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) { > + return; > + } > + > + if (!props.is_socket || !props.is_unix) { > + error_setg(errp, "chardev \"%s\" is not a unix socket", > + s->sec_indev); I believe tcp socket is also supported? > + return; > + } > + > + s->chr_out = qemu_chr_find(s->outdev); > + if (s->chr_out == NULL) { > + error_setg(errp, "OUT Device '%s' not found", s->outdev); > + return; > + } > + > + memset(&props, 0, sizeof(props)); > + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) { > + return; > + } > + > + if (!props.is_socket || !props.is_unix) { > + error_setg(errp, "chardev \"%s\" is not a unix socket", > + s->outdev); Ditto, and there's code duplication, please introduce a helper to do above. > + return; > + } > + > + qemu_chr_fe_claim_no_fail(s->chr_pri_in); > + > + qemu_chr_fe_claim_no_fail(s->chr_sec_in); > + > + qemu_chr_fe_claim_no_fail(s->chr_out); > + > + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); > + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); > + > + return; > +} > + > +static void colo_compare_class_init(ObjectClass *oc, void *data) > +{ > + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); > + > + ucc->complete = colo_compare_complete; > +} > + > +static void colo_compare_init(Object *obj) > +{ > + object_property_add_str(obj, "primary_in", > + compare_get_pri_indev, compare_set_pri_indev, > + NULL); > + object_property_add_str(obj, "secondary_in", > + compare_get_sec_indev, compare_set_sec_indev, > + NULL); > + object_property_add_str(obj, "outdev", > + compare_get_outdev, compare_set_outdev, > + NULL); > +} > + > +static void colo_compare_finalize(Object *obj) > +{ > + CompareState *s = COLO_COMPARE(obj); > + > + if (s->chr_pri_in) { > + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); > + qemu_chr_fe_release(s->chr_pri_in); > + } > + if (s->chr_sec_in) { > + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); > + qemu_chr_fe_release(s->chr_sec_in); > + } > + if (s->chr_out) { > + qemu_chr_fe_release(s->chr_out); > + } > + > + g_free(s->pri_indev); > + g_free(s->sec_indev); > + g_free(s->outdev); > +} > + > +static const TypeInfo colo_compare_info = { > + .name = TYPE_COLO_COMPARE, > + .parent = TYPE_OBJECT, > + .instance_size = sizeof(CompareState), > + .instance_init = colo_compare_init, > + .instance_finalize = colo_compare_finalize, > + .class_size = sizeof(CompareClass), > + .class_init = colo_compare_class_init, > + .interfaces = (InterfaceInfo[]) { > + { TYPE_USER_CREATABLE }, > + { } > + } > +}; > + > +static void register_types(void) > +{ > + type_register_static(&colo_compare_info); > +} > + > +type_init(register_types); > diff --git a/qemu-options.hx b/qemu-options.hx > index 587de8f..33d5d0b 100644 > --- a/qemu-options.hx > +++ b/qemu-options.hx > @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by > The file format is libpcap, so it can be analyzed with tools such as tcpdump > or Wireshark. > > +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, > +outdev=@var{chardevid} > + > +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with > +secondary packet. If the packet same, we will output primary s/If the packet same/If the packets are same/. > +packet to outdev@var{chardevid}, else we will notify colo-frame > +do checkpoint and send primary packet to outdev@var{chardevid}. > + > +we can use it with the help of filter-mirror and filter-redirector. s/we/We/ and looks like colo compare must be used with the help of mirror and redirector? > + > +@example > + > +primary: > +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown > +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 > +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait > +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait > +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait > +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 > +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait > +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 > +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 > +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out > +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 > +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 > + > +secondary: > +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown > +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 > +-chardev socket,id=red0,host=3.3.3.3,port=9003 > +-chardev socket,id=red1,host=3.3.3.3,port=9004 > +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 > +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 > + > +@end example > + > +If you want to know the detail of above command line, you can read > +the colo-compare git log. > + > @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] > @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] > > diff --git a/vl.c b/vl.c > index cbe51ac..c6b9a6f 100644 > --- a/vl.c > +++ b/vl.c > @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) > if (g_str_equal(type, "filter-buffer") || > g_str_equal(type, "filter-dump") || > g_str_equal(type, "filter-mirror") || > - g_str_equal(type, "filter-redirector")) { > + g_str_equal(type, "filter-redirector") || > + g_str_equal(type, "colo-compare")) { > return false; > } >
On 2016/8/31 15:53, Jason Wang wrote: > > > On 2016年08月17日 16:10, Zhang Chen wrote: >> This a COLO net ascii figure: >> >> Primary qemu Secondary qemu >> +--------------------------------------------------------------+ +----------------------------------------------------------------+ >> | +----------------------------------------------------------+ | | +-----------------------------------------------------------+ | >> | | | | | | | | >> | | guest | | | | guest | | >> | | | | | | | | >> | +-------^--------------------------+-----------------------+ | | +---------------------+--------+----------------------------+ | >> | | | | | ^ | | >> | | | | | | | | >> | | +------------------------------------------------------+ | | | | >> |netfilter| | | | | | netfilter | | | >> | +----------+ +----------------------------+ | | | +-----------------------------------------------------------+ | >> | | | | | | out | | | | | | filter excute order | | >> | | | | +-----------------------------+ | | | | | | +-------------------> | | >> | | | | | | | | | | | | | | TCP | | >> | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| | | | +------------+ +---+----+---v+rewriter++ +------------+ | | >> | | | | | | | | |in | |in | | | | | | | | | | | | | >> | | | filter | | filter | | filter +------> colo <------+ +--------> filter +--> adjust | adjust +--> filter | | | >> | | | mirror | |redirector| |redirector| | | compare | | | | | | redirector | | ack | seq | | redirector | | | >> | | | | | | | | | | | | | | | | | | | | | | | | >> | | +----^-----+ +----+-----+ +----------+ | +---------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | >> | | | tx | rx rx | | | | | tx all | rx | | >> | | | | | | | | +-----------------------------------------------------------+ | >> | | | +--------------+ | | | | | | >> | | | filter excute order | | | | | | | >> | | | +----------------> | | | +--------------------------------------------------------+ | >> | +-----------------------------------------+ | | | >> | | | | | | >> +--------------------------------------------------------------+ +----------------------------------------------------------------+ >> |guest receive | guest send >> | | >> +--------+----------------------------v------------------------+ >> | | NOTE: filter direction is rx/tx/all >> | tap | rx:receive packets sent to the netdev >> | | tx:receive packets sent by the netdev >> +--------------------------------------------------------------+ > > It's better to add a doc under docs to explain this configuration in > detail on top of this series. > Agreed! I'm adding a new document to introduce COLO (COLO-FT.txt). Since COLO proxy is only used in COLO, and IMHO, it is unnecessary to add a special documentation to describe this, we can add it into the COLO-FT.txt after it is been merged ... >> In COLO-compare, we do packet comparing job. >> Packets coming from the primary char indev will be sent to outdev. >> Packets coming from the secondary char dev will be dropped after comparing. >> colo-comapre need two input chardev and one output chardev: >> primary_in=chardev1-id (source: primary send packet) >> secondary_in=chardev2-id (source: secondary send packet) >> outdev=chardev3-id >> >> usage: >> >> primary: >> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> >> secondary: >> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown >> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=red0,host=3.3.3.3,port=9003 >> -chardev socket,id=red1,host=3.3.3.3,port=9004 >> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> >> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >> --- >> net/Makefile.objs | 1 + >> net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> qemu-options.hx | 39 ++++++++ >> vl.c | 3 +- >> 4 files changed, 326 insertions(+), 1 deletion(-) >> create mode 100644 net/colo-compare.c >> >> diff --git a/net/Makefile.objs b/net/Makefile.objs >> index b7c22fd..ba92f73 100644 >> --- a/net/Makefile.objs >> +++ b/net/Makefile.objs >> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >> common-obj-y += filter.o >> common-obj-y += filter-buffer.o >> common-obj-y += filter-mirror.o >> +common-obj-y += colo-compare.o >> diff --git a/net/colo-compare.c b/net/colo-compare.c >> new file mode 100644 >> index 0000000..cdc3e0e >> --- /dev/null >> +++ b/net/colo-compare.c >> @@ -0,0 +1,284 @@ >> +/* >> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) >> + * (a.k.a. Fault Tolerance or Continuous Replication) >> + * >> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >> + * Copyright (c) 2016 FUJITSU LIMITED >> + * Copyright (c) 2016 Intel Corporation >> + * >> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> + * >> + * This work is licensed under the terms of the GNU GPL, version 2 or >> + * later. See the COPYING file in the top-level directory. >> + */ >> + >> +#include "qemu/osdep.h" >> +#include "qemu/error-report.h" >> +#include "qemu-common.h" >> +#include "qapi/qmp/qerror.h" >> +#include "qapi/error.h" >> +#include "net/net.h" >> +#include "net/vhost_net.h" > > Looks unnecessary. > >> +#include "qom/object_interfaces.h" >> +#include "qemu/iov.h" >> +#include "qom/object.h" >> +#include "qemu/typedefs.h" >> +#include "net/queue.h" >> +#include "sysemu/char.h" >> +#include "qemu/sockets.h" >> +#include "qapi-visit.h" >> + >> +#define TYPE_COLO_COMPARE "colo-compare" >> +#define COLO_COMPARE(obj) \ >> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >> + >> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >> + >> +typedef struct CompareState { >> + Object parent; >> + >> + char *pri_indev; >> + char *sec_indev; >> + char *outdev; >> + CharDriverState *chr_pri_in; >> + CharDriverState *chr_sec_in; >> + CharDriverState *chr_out; >> + QTAILQ_ENTRY(CompareState) next; > > This looks not used in this series but in commit "colo-compare and > filter-rewriter work with colo-frame". We'd better delay the introducing > to that patch. > >> + SocketReadState pri_rs; >> + SocketReadState sec_rs; >> +} CompareState; >> + >> +typedef struct CompareClass { >> + ObjectClass parent_class; >> +} CompareClass; >> + >> +typedef struct CompareChardevProps { >> + bool is_socket; >> + bool is_unix; >> +} CompareChardevProps; >> + >> +static char *compare_get_pri_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->pri_indev); >> +} >> + >> +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->pri_indev); >> + s->pri_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_sec_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->sec_indev); >> +} >> + >> +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->sec_indev); >> + s->sec_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_outdev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->outdev); >> +} >> + >> +static void compare_set_outdev(Object *obj, const char *value, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->outdev); >> + s->outdev = g_strdup(value); >> +} >> + >> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >> +{ >> + /* if packet_enqueue pri pkt failed we will send unsupported packet */ >> +} >> + >> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >> +{ >> + /* if packet_enqueue sec pkt failed we will notify trace */ >> +} >> + >> +static int compare_chardev_opts(void *opaque, >> + const char *name, const char *value, >> + Error **errp) >> +{ >> + CompareChardevProps *props = opaque; >> + >> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { >> + props->is_socket = true; >> + } else if (strcmp(name, "host") == 0) { > > Typo? net_vhost_chardev_opts() did: > > } else if (strcmp(name, "path") == 0) { > props->is_unix = true; > } > > > >> + props->is_unix = true; >> + } else if (strcmp(name, "port") == 0) { >> + } else if (strcmp(name, "server") == 0) { >> + } else if (strcmp(name, "wait") == 0) { >> + } else { >> + error_setg(errp, >> + "COLO-compare does not support a chardev with option %s=%s", >> + name, value); >> + return -1; >> + } >> + return 0; >> +} >> + >> +/* >> + * called from the main thread on the primary >> + * to setup colo-compare. >> + */ >> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(uc); >> + CompareChardevProps props; >> + >> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >> + error_setg(errp, "colo compare needs 'primary_in' ," >> + "'secondary_in','outdev' property set"); >> + return; >> + } else if (!strcmp(s->pri_indev, s->outdev) || >> + !strcmp(s->sec_indev, s->outdev) || >> + !strcmp(s->pri_indev, s->sec_indev)) { >> + error_setg(errp, "'indev' and 'outdev' could not be same " >> + "for compare module"); >> + return; >> + } >> + >> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >> + if (s->chr_pri_in == NULL) { >> + error_setg(errp, "Primary IN Device '%s' not found", >> + s->pri_indev); >> + return; >> + } >> + >> + /* inspect chardev opts */ >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->pri_indev); >> + return; >> + } >> + >> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >> + if (s->chr_sec_in == NULL) { >> + error_setg(errp, "Secondary IN Device '%s' not found", >> + s->sec_indev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->sec_indev); > > I believe tcp socket is also supported? > >> + return; >> + } >> + >> + s->chr_out = qemu_chr_find(s->outdev); >> + if (s->chr_out == NULL) { >> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->outdev); > > Ditto, and there's code duplication, please introduce a helper to do above. > >> + return; >> + } >> + >> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_out); >> + >> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >> + >> + return; >> +} >> + >> +static void colo_compare_class_init(ObjectClass *oc, void *data) >> +{ >> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >> + >> + ucc->complete = colo_compare_complete; >> +} >> + >> +static void colo_compare_init(Object *obj) >> +{ >> + object_property_add_str(obj, "primary_in", >> + compare_get_pri_indev, compare_set_pri_indev, >> + NULL); >> + object_property_add_str(obj, "secondary_in", >> + compare_get_sec_indev, compare_set_sec_indev, >> + NULL); >> + object_property_add_str(obj, "outdev", >> + compare_get_outdev, compare_set_outdev, >> + NULL); >> +} >> + >> +static void colo_compare_finalize(Object *obj) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + if (s->chr_pri_in) { >> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_pri_in); >> + } >> + if (s->chr_sec_in) { >> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_sec_in); >> + } >> + if (s->chr_out) { >> + qemu_chr_fe_release(s->chr_out); >> + } >> + >> + g_free(s->pri_indev); >> + g_free(s->sec_indev); >> + g_free(s->outdev); >> +} >> + >> +static const TypeInfo colo_compare_info = { >> + .name = TYPE_COLO_COMPARE, >> + .parent = TYPE_OBJECT, >> + .instance_size = sizeof(CompareState), >> + .instance_init = colo_compare_init, >> + .instance_finalize = colo_compare_finalize, >> + .class_size = sizeof(CompareClass), >> + .class_init = colo_compare_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { TYPE_USER_CREATABLE }, >> + { } >> + } >> +}; >> + >> +static void register_types(void) >> +{ >> + type_register_static(&colo_compare_info); >> +} >> + >> +type_init(register_types); >> diff --git a/qemu-options.hx b/qemu-options.hx >> index 587de8f..33d5d0b 100644 >> --- a/qemu-options.hx >> +++ b/qemu-options.hx >> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by >> The file format is libpcap, so it can be analyzed with tools such as tcpdump >> or Wireshark. >> >> +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >> +outdev=@var{chardevid} >> + >> +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with >> +secondary packet. If the packet same, we will output primary > > s/If the packet same/If the packets are same/. > >> +packet to outdev@var{chardevid}, else we will notify colo-frame >> +do checkpoint and send primary packet to outdev@var{chardevid}. >> + >> +we can use it with the help of filter-mirror and filter-redirector. > > s/we/We/ and looks like colo compare must be used with the help of > mirror and redirector? > >> + >> +@example >> + >> +primary: >> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> + >> +secondary: >> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown >> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> + >> +@end example >> + >> +If you want to know the detail of above command line, you can read >> +the colo-compare git log. >> + >> @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> >> diff --git a/vl.c b/vl.c >> index cbe51ac..c6b9a6f 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) >> if (g_str_equal(type, "filter-buffer") || >> g_str_equal(type, "filter-dump") || >> g_str_equal(type, "filter-mirror") || >> - g_str_equal(type, "filter-redirector")) { >> + g_str_equal(type, "filter-redirector") || >> + g_str_equal(type, "colo-compare")) { >> return false; >> } >> > > > . >
On 08/31/2016 03:53 PM, Jason Wang wrote: > > > On 2016年08月17日 16:10, Zhang Chen wrote: >> This a COLO net ascii figure: >> >> Primary qemu Secondary qemu >> +--------------------------------------------------------------+ >> +----------------------------------------------------------------+ >> | +----------------------------------------------------------+ >> | | >> +-----------------------------------------------------------+ | >> | | | >> | | | >> | | >> | | guest | >> | | | guest | | >> | | | >> | | | >> | | >> | +-------^--------------------------+-----------------------+ >> | | >> +---------------------+--------+----------------------------+ | >> | | | | >> | ^ | | >> | | | | >> | | | | >> | | +------------------------------------------------------+ >> | | | | >> |netfilter| | | | | | >> netfilter | | | >> | +----------+ +----------------------------+ | | | >> +-----------------------------------------------------------+ | >> | | | | | | out | | | >> | | | filter excute order | | >> | | | | +-----------------------------+ | | | >> | | | +-------------------> | | >> | | | | | | | | | | | >> | | | TCP | | >> | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| >> | | | +------------+ +---+----+---v+rewriter++ +------------+ | | >> | | | | | | | | |in | |in | >> | | | | | | | | | | | | >> | | | filter | | filter | | filter +------> colo <------+ >> +--------> filter +--> adjust | adjust +--> filter | | | >> | | | mirror | |redirector| |redirector| | | compare | | | >> | | | redirector | | ack | seq | | redirector | | | >> | | | | | | | | | | | | | >> | | | | | | | | | | | >> | | +----^-----+ +----+-----+ +----------+ | +---------+ | | >> | | +------------+ +--------+--------------+ +---+--------+ | | >> | | | tx | rx rx | | | | | >> tx all | rx | | >> | | | | | | | | >> +-----------------------------------------------------------+ | >> | | | +--------------+ | | | | | | >> | | | filter excute order | | | | | | | >> | | | +----------------> | | | >> +--------------------------------------------------------+ | >> | +-----------------------------------------+ | | | >> | | | | | | >> +--------------------------------------------------------------+ >> +----------------------------------------------------------------+ >> |guest receive | guest send >> | | >> +--------+----------------------------v------------------------+ >> | | NOTE: filter direction is rx/tx/all >> | tap | rx:receive >> packets sent to the netdev >> | | tx:receive packets sent by the netdev >> +--------------------------------------------------------------+ > > It's better to add a doc under docs to explain this configuration in > detail on top of this series. > As you say, Am I add /docs/colo-proxy.txt to explain it or add this in hailiang's COLO-FT.txt after merge? >> In COLO-compare, we do packet comparing job. >> Packets coming from the primary char indev will be sent to outdev. >> Packets coming from the secondary char dev will be dropped after >> comparing. >> colo-comapre need two input chardev and one output chardev: >> primary_in=chardev1-id (source: primary send packet) >> secondary_in=chardev2-id (source: secondary send packet) >> outdev=chardev3-id >> >> usage: >> >> primary: >> -netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> -object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> -object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> >> secondary: >> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=red0,host=3.3.3.3,port=9003 >> -chardev socket,id=red1,host=3.3.3.3,port=9004 >> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> >> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >> --- >> net/Makefile.objs | 1 + >> net/colo-compare.c | 284 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> qemu-options.hx | 39 ++++++++ >> vl.c | 3 +- >> 4 files changed, 326 insertions(+), 1 deletion(-) >> create mode 100644 net/colo-compare.c >> >> diff --git a/net/Makefile.objs b/net/Makefile.objs >> index b7c22fd..ba92f73 100644 >> --- a/net/Makefile.objs >> +++ b/net/Makefile.objs >> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >> common-obj-y += filter.o >> common-obj-y += filter-buffer.o >> common-obj-y += filter-mirror.o >> +common-obj-y += colo-compare.o >> diff --git a/net/colo-compare.c b/net/colo-compare.c >> new file mode 100644 >> index 0000000..cdc3e0e >> --- /dev/null >> +++ b/net/colo-compare.c >> @@ -0,0 +1,284 @@ >> +/* >> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service >> (COLO) >> + * (a.k.a. Fault Tolerance or Continuous Replication) >> + * >> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >> + * Copyright (c) 2016 FUJITSU LIMITED >> + * Copyright (c) 2016 Intel Corporation >> + * >> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >> + * >> + * This work is licensed under the terms of the GNU GPL, version 2 or >> + * later. See the COPYING file in the top-level directory. >> + */ >> + >> +#include "qemu/osdep.h" >> +#include "qemu/error-report.h" >> +#include "qemu-common.h" >> +#include "qapi/qmp/qerror.h" >> +#include "qapi/error.h" >> +#include "net/net.h" >> +#include "net/vhost_net.h" > > Looks unnecessary. I will remove it. > >> +#include "qom/object_interfaces.h" >> +#include "qemu/iov.h" >> +#include "qom/object.h" >> +#include "qemu/typedefs.h" >> +#include "net/queue.h" >> +#include "sysemu/char.h" >> +#include "qemu/sockets.h" >> +#include "qapi-visit.h" >> + >> +#define TYPE_COLO_COMPARE "colo-compare" >> +#define COLO_COMPARE(obj) \ >> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >> + >> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >> + >> +typedef struct CompareState { >> + Object parent; >> + >> + char *pri_indev; >> + char *sec_indev; >> + char *outdev; >> + CharDriverState *chr_pri_in; >> + CharDriverState *chr_sec_in; >> + CharDriverState *chr_out; >> + QTAILQ_ENTRY(CompareState) next; > > This looks not used in this series but in commit "colo-compare and > filter-rewriter work with colo-frame". We'd better delay the > introducing to that patch. OK~ I got your point. > >> + SocketReadState pri_rs; >> + SocketReadState sec_rs; >> +} CompareState; >> + >> +typedef struct CompareClass { >> + ObjectClass parent_class; >> +} CompareClass; >> + >> +typedef struct CompareChardevProps { >> + bool is_socket; >> + bool is_unix; >> +} CompareChardevProps; >> + >> +static char *compare_get_pri_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->pri_indev); >> +} >> + >> +static void compare_set_pri_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->pri_indev); >> + s->pri_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_sec_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->sec_indev); >> +} >> + >> +static void compare_set_sec_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->sec_indev); >> + s->sec_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_outdev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->outdev); >> +} >> + >> +static void compare_set_outdev(Object *obj, const char *value, Error >> **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->outdev); >> + s->outdev = g_strdup(value); >> +} >> + >> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >> +{ >> + /* if packet_enqueue pri pkt failed we will send unsupported >> packet */ >> +} >> + >> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >> +{ >> + /* if packet_enqueue sec pkt failed we will notify trace */ >> +} >> + >> +static int compare_chardev_opts(void *opaque, >> + const char *name, const char *value, >> + Error **errp) >> +{ >> + CompareChardevProps *props = opaque; >> + >> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { >> + props->is_socket = true; >> + } else if (strcmp(name, "host") == 0) { > > Typo? net_vhost_chardev_opts() did: > > } else if (strcmp(name, "path") == 0) { > props->is_unix = true; > } > > No, In colo-compare we use chardev like this: -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait If we only use "path" here will trigger a error. Should I add anthor "path" here? > >> + props->is_unix = true; >> + } else if (strcmp(name, "port") == 0) { >> + } else if (strcmp(name, "server") == 0) { >> + } else if (strcmp(name, "wait") == 0) { >> + } else { >> + error_setg(errp, >> + "COLO-compare does not support a chardev with >> option %s=%s", >> + name, value); >> + return -1; >> + } >> + return 0; >> +} >> + >> +/* >> + * called from the main thread on the primary >> + * to setup colo-compare. >> + */ >> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(uc); >> + CompareChardevProps props; >> + >> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >> + error_setg(errp, "colo compare needs 'primary_in' ," >> + "'secondary_in','outdev' property set"); >> + return; >> + } else if (!strcmp(s->pri_indev, s->outdev) || >> + !strcmp(s->sec_indev, s->outdev) || >> + !strcmp(s->pri_indev, s->sec_indev)) { >> + error_setg(errp, "'indev' and 'outdev' could not be same " >> + "for compare module"); >> + return; >> + } >> + >> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >> + if (s->chr_pri_in == NULL) { >> + error_setg(errp, "Primary IN Device '%s' not found", >> + s->pri_indev); >> + return; >> + } >> + >> + /* inspect chardev opts */ >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->pri_indev); >> + return; >> + } >> + >> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >> + if (s->chr_sec_in == NULL) { >> + error_setg(errp, "Secondary IN Device '%s' not found", >> + s->sec_indev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->sec_indev); > > I believe tcp socket is also supported? If I understand correctly, "tcp socket" in here is the "-chardev socket". I will rename "unix socket" to "tcp socket". > >> + return; >> + } >> + >> + s->chr_out = qemu_chr_find(s->outdev); >> + if (s->chr_out == NULL) { >> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->outdev); > > Ditto, and there's code duplication, please introduce a helper to do > above. I don't understand what the "helper"? In here we check each chardev, will I change to "goto error;" ? > >> + return; >> + } >> + >> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_out); >> + >> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >> + >> + return; >> +} >> + >> +static void colo_compare_class_init(ObjectClass *oc, void *data) >> +{ >> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >> + >> + ucc->complete = colo_compare_complete; >> +} >> + >> +static void colo_compare_init(Object *obj) >> +{ >> + object_property_add_str(obj, "primary_in", >> + compare_get_pri_indev, >> compare_set_pri_indev, >> + NULL); >> + object_property_add_str(obj, "secondary_in", >> + compare_get_sec_indev, >> compare_set_sec_indev, >> + NULL); >> + object_property_add_str(obj, "outdev", >> + compare_get_outdev, compare_set_outdev, >> + NULL); >> +} >> + >> +static void colo_compare_finalize(Object *obj) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + if (s->chr_pri_in) { >> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_pri_in); >> + } >> + if (s->chr_sec_in) { >> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_sec_in); >> + } >> + if (s->chr_out) { >> + qemu_chr_fe_release(s->chr_out); >> + } >> + >> + g_free(s->pri_indev); >> + g_free(s->sec_indev); >> + g_free(s->outdev); >> +} >> + >> +static const TypeInfo colo_compare_info = { >> + .name = TYPE_COLO_COMPARE, >> + .parent = TYPE_OBJECT, >> + .instance_size = sizeof(CompareState), >> + .instance_init = colo_compare_init, >> + .instance_finalize = colo_compare_finalize, >> + .class_size = sizeof(CompareClass), >> + .class_init = colo_compare_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { TYPE_USER_CREATABLE }, >> + { } >> + } >> +}; >> + >> +static void register_types(void) >> +{ >> + type_register_static(&colo_compare_info); >> +} >> + >> +type_init(register_types); >> diff --git a/qemu-options.hx b/qemu-options.hx >> index 587de8f..33d5d0b 100644 >> --- a/qemu-options.hx >> +++ b/qemu-options.hx >> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} >> to the file specified by >> The file format is libpcap, so it can be analyzed with tools such >> as tcpdump >> or Wireshark. >> +@item -object >> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >> +outdev=@var{chardevid} >> + >> +Colo-compare gets packet from primary_in@var{chardevid} and >> secondary_in@var{chardevid}, than compare primary packet with >> +secondary packet. If the packet same, we will output primary > > s/If the packet same/If the packets are same/. OK. > >> +packet to outdev@var{chardevid}, else we will notify colo-frame >> +do checkpoint and send primary packet to outdev@var{chardevid}. >> + >> +we can use it with the help of filter-mirror and filter-redirector. > > s/we/We/ and looks like colo compare must be used with the help of > mirror and redirector? Currently yes. > >> + >> +@example >> + >> +primary: >> +-netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> +-object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> +-object >> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> +-object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> + >> +secondary: >> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> + >> +@end example >> + >> +If you want to know the detail of above command line, you can read >> +the colo-compare git log. >> + >> @item -object >> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> @item -object >> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> diff --git a/vl.c b/vl.c >> index cbe51ac..c6b9a6f 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >> *type) >> if (g_str_equal(type, "filter-buffer") || >> g_str_equal(type, "filter-dump") || >> g_str_equal(type, "filter-mirror") || >> - g_str_equal(type, "filter-redirector")) { >> + g_str_equal(type, "filter-redirector") || >> + g_str_equal(type, "colo-compare")) { >> return false; >> } > > > > . >
On 2016年08月31日 17:03, Zhang Chen wrote: > > > On 08/31/2016 03:53 PM, Jason Wang wrote: >> >> >> On 2016年08月17日 16:10, Zhang Chen wrote: >>> This a COLO net ascii figure: >>> >>> Primary qemu Secondary qemu >>> +--------------------------------------------------------------+ >>> +----------------------------------------------------------------+ >>> | +----------------------------------------------------------+ >>> | | >>> +-----------------------------------------------------------+ | >>> | | | >>> | | >>> | | | >>> | | guest | >>> | | | guest | | >>> | | | >>> | | >>> | | | >>> | +-------^--------------------------+-----------------------+ >>> | | >>> +---------------------+--------+----------------------------+ | >>> | | | | | ^ >>> | | >>> | | | | | | >>> | | >>> | | +------------------------------------------------------+ >>> | | | | >>> |netfilter| | | | | | >>> netfilter | | | >>> | +----------+ +----------------------------+ | | | >>> +-----------------------------------------------------------+ | >>> | | | | | | out | | | >>> | | | filter excute order | | >>> | | | | +-----------------------------+ | | | >>> | | | +-------------------> | | >>> | | | | | | | | | | | >>> | | | TCP | | >>> | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| >>> | | | +------------+ +---+----+---v+rewriter++ +------------+ | | >>> | | | | | | | | |in | |in | >>> | | | | | | | | | | | | >>> | | | filter | | filter | | filter +------> colo <------+ >>> +--------> filter +--> adjust | adjust +--> filter | | | >>> | | | mirror | |redirector| |redirector| | | compare | | | >>> | | | redirector | | ack | seq | | redirector | | | >>> | | | | | | | | | | | | | >>> | | | | | | | | | | | >>> | | +----^-----+ +----+-----+ +----------+ | +---------+ | | >>> | | +------------+ +--------+--------------+ +---+--------+ | | >>> | | | tx | rx rx | | | | | >>> tx all | rx | | >>> | | | | | | | | >>> +-----------------------------------------------------------+ | >>> | | | +--------------+ | | | | | | >>> | | | filter excute order | | | | | | | >>> | | | +----------------> | | | >>> +--------------------------------------------------------+ | >>> | +-----------------------------------------+ | | | >>> | | | | | | >>> +--------------------------------------------------------------+ >>> +----------------------------------------------------------------+ >>> |guest receive | guest send >>> | | >>> +--------+----------------------------v------------------------+ >>> | | NOTE: filter direction is rx/tx/all >>> | tap | rx:receive packets sent to the netdev >>> | | tx:receive packets sent by the netdev >>> +--------------------------------------------------------------+ >> >> It's better to add a doc under docs to explain this configuration in >> detail on top of this series. >> > > As you say, Am I add /docs/colo-proxy.txt to explain it or add this in > hailiang's COLO-FT.txt after merge? > >>> In COLO-compare, we do packet comparing job. >>> Packets coming from the primary char indev will be sent to outdev. >>> Packets coming from the secondary char dev will be dropped after >>> comparing. >>> colo-comapre need two input chardev and one output chardev: >>> primary_in=chardev1-id (source: primary send packet) >>> secondary_in=chardev2-id (source: secondary send packet) >>> outdev=chardev3-id >>> >>> usage: >>> >>> primary: >>> -netdev >>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>> -object >>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>> -object >>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>> -object >>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>> >>> secondary: >>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>> script=/etc/qemu-ifdown >>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>> -chardev socket,id=red0,host=3.3.3.3,port=9003 >>> -chardev socket,id=red1,host=3.3.3.3,port=9004 >>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>> >>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >>> --- >>> net/Makefile.objs | 1 + >>> net/colo-compare.c | 284 >>> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> qemu-options.hx | 39 ++++++++ >>> vl.c | 3 +- >>> 4 files changed, 326 insertions(+), 1 deletion(-) >>> create mode 100644 net/colo-compare.c >>> >>> diff --git a/net/Makefile.objs b/net/Makefile.objs >>> index b7c22fd..ba92f73 100644 >>> --- a/net/Makefile.objs >>> +++ b/net/Makefile.objs >>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >>> common-obj-y += filter.o >>> common-obj-y += filter-buffer.o >>> common-obj-y += filter-mirror.o >>> +common-obj-y += colo-compare.o >>> diff --git a/net/colo-compare.c b/net/colo-compare.c >>> new file mode 100644 >>> index 0000000..cdc3e0e >>> --- /dev/null >>> +++ b/net/colo-compare.c >>> @@ -0,0 +1,284 @@ >>> +/* >>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service >>> (COLO) >>> + * (a.k.a. Fault Tolerance or Continuous Replication) >>> + * >>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >>> + * Copyright (c) 2016 FUJITSU LIMITED >>> + * Copyright (c) 2016 Intel Corporation >>> + * >>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>> + * >>> + * This work is licensed under the terms of the GNU GPL, version 2 or >>> + * later. See the COPYING file in the top-level directory. >>> + */ >>> + >>> +#include "qemu/osdep.h" >>> +#include "qemu/error-report.h" >>> +#include "qemu-common.h" >>> +#include "qapi/qmp/qerror.h" >>> +#include "qapi/error.h" >>> +#include "net/net.h" >>> +#include "net/vhost_net.h" >> >> Looks unnecessary. > > I will remove it. > >> >>> +#include "qom/object_interfaces.h" >>> +#include "qemu/iov.h" >>> +#include "qom/object.h" >>> +#include "qemu/typedefs.h" >>> +#include "net/queue.h" >>> +#include "sysemu/char.h" >>> +#include "qemu/sockets.h" >>> +#include "qapi-visit.h" >>> + >>> +#define TYPE_COLO_COMPARE "colo-compare" >>> +#define COLO_COMPARE(obj) \ >>> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >>> + >>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >>> + >>> +typedef struct CompareState { >>> + Object parent; >>> + >>> + char *pri_indev; >>> + char *sec_indev; >>> + char *outdev; >>> + CharDriverState *chr_pri_in; >>> + CharDriverState *chr_sec_in; >>> + CharDriverState *chr_out; >>> + QTAILQ_ENTRY(CompareState) next; >> >> This looks not used in this series but in commit "colo-compare and >> filter-rewriter work with colo-frame". We'd better delay the >> introducing to that patch. > > OK~ I got your point. > >> >>> + SocketReadState pri_rs; >>> + SocketReadState sec_rs; >>> +} CompareState; >>> + >>> +typedef struct CompareClass { >>> + ObjectClass parent_class; >>> +} CompareClass; >>> + >>> +typedef struct CompareChardevProps { >>> + bool is_socket; >>> + bool is_unix; >>> +} CompareChardevProps; >>> + >>> +static char *compare_get_pri_indev(Object *obj, Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + return g_strdup(s->pri_indev); >>> +} >>> + >>> +static void compare_set_pri_indev(Object *obj, const char *value, >>> Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + g_free(s->pri_indev); >>> + s->pri_indev = g_strdup(value); >>> +} >>> + >>> +static char *compare_get_sec_indev(Object *obj, Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + return g_strdup(s->sec_indev); >>> +} >>> + >>> +static void compare_set_sec_indev(Object *obj, const char *value, >>> Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + g_free(s->sec_indev); >>> + s->sec_indev = g_strdup(value); >>> +} >>> + >>> +static char *compare_get_outdev(Object *obj, Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + return g_strdup(s->outdev); >>> +} >>> + >>> +static void compare_set_outdev(Object *obj, const char *value, >>> Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + g_free(s->outdev); >>> + s->outdev = g_strdup(value); >>> +} >>> + >>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >>> +{ >>> + /* if packet_enqueue pri pkt failed we will send unsupported >>> packet */ >>> +} >>> + >>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >>> +{ >>> + /* if packet_enqueue sec pkt failed we will notify trace */ >>> +} >>> + >>> +static int compare_chardev_opts(void *opaque, >>> + const char *name, const char *value, >>> + Error **errp) >>> +{ >>> + CompareChardevProps *props = opaque; >>> + >>> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == >>> 0) { >>> + props->is_socket = true; >>> + } else if (strcmp(name, "host") == 0) { >> >> Typo? net_vhost_chardev_opts() did: >> >> } else if (strcmp(name, "path") == 0) { >> props->is_unix = true; >> } >> >> > > No, In colo-compare we use chardev like this: > > -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait > > If we only use "path" here will trigger a error. > Should I add anthor "path" here? If I understand the code correctly, "is_unix" means "is unix domain socket"? If yes, according to the help: -chardev socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds] [,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off] [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp) -chardev socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds] [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix) "host" will not be used for UNIX domain socket. And if UNIX domain socket is not supported, there's probably no need to differentiate it from other types. > > >> >>> + props->is_unix = true; >>> + } else if (strcmp(name, "port") == 0) { >>> + } else if (strcmp(name, "server") == 0) { >>> + } else if (strcmp(name, "wait") == 0) { >>> + } else { >>> + error_setg(errp, >>> + "COLO-compare does not support a chardev with >>> option %s=%s", >>> + name, value); >>> + return -1; >>> + } >>> + return 0; >>> +} >>> + >>> +/* >>> + * called from the main thread on the primary >>> + * to setup colo-compare. >>> + */ >>> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >>> +{ >>> + CompareState *s = COLO_COMPARE(uc); >>> + CompareChardevProps props; >>> + >>> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >>> + error_setg(errp, "colo compare needs 'primary_in' ," >>> + "'secondary_in','outdev' property set"); >>> + return; >>> + } else if (!strcmp(s->pri_indev, s->outdev) || >>> + !strcmp(s->sec_indev, s->outdev) || >>> + !strcmp(s->pri_indev, s->sec_indev)) { >>> + error_setg(errp, "'indev' and 'outdev' could not be same " >>> + "for compare module"); >>> + return; >>> + } >>> + >>> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >>> + if (s->chr_pri_in == NULL) { >>> + error_setg(errp, "Primary IN Device '%s' not found", >>> + s->pri_indev); >>> + return; >>> + } >>> + >>> + /* inspect chardev opts */ >>> + memset(&props, 0, sizeof(props)); >>> + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, >>> &props, errp)) { >>> + return; >>> + } >>> + >>> + if (!props.is_socket || !props.is_unix) { >>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>> + s->pri_indev); >>> + return; >>> + } >>> + >>> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >>> + if (s->chr_sec_in == NULL) { >>> + error_setg(errp, "Secondary IN Device '%s' not found", >>> + s->sec_indev); >>> + return; >>> + } >>> + >>> + memset(&props, 0, sizeof(props)); >>> + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, >>> &props, errp)) { >>> + return; >>> + } >>> + >>> + if (!props.is_socket || !props.is_unix) { >>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>> + s->sec_indev); >> >> I believe tcp socket is also supported? > > If I understand correctly, "tcp socket" in here is the "-chardev socket". > I will rename "unix socket" to "tcp socket". > >> >>> + return; >>> + } >>> + >>> + s->chr_out = qemu_chr_find(s->outdev); >>> + if (s->chr_out == NULL) { >>> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >>> + return; >>> + } >>> + >>> + memset(&props, 0, sizeof(props)); >>> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, >>> &props, errp)) { >>> + return; >>> + } >>> + >>> + if (!props.is_socket || !props.is_unix) { >>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>> + s->outdev); >> >> Ditto, and there's code duplication, please introduce a helper to do >> above. > > I don't understand what the "helper"? > In here we check each chardev, will I change to "goto error;" ? A helper to avoid the code duplication for socket type inspection for pri_in,scr_in and chr_out. > >> >>> + return; >>> + } >>> + >>> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >>> + >>> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >>> + >>> + qemu_chr_fe_claim_no_fail(s->chr_out); >>> + >>> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >>> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >>> + >>> + return; >>> +} >>> + >>> +static void colo_compare_class_init(ObjectClass *oc, void *data) >>> +{ >>> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >>> + >>> + ucc->complete = colo_compare_complete; >>> +} >>> + >>> +static void colo_compare_init(Object *obj) >>> +{ >>> + object_property_add_str(obj, "primary_in", >>> + compare_get_pri_indev, >>> compare_set_pri_indev, >>> + NULL); >>> + object_property_add_str(obj, "secondary_in", >>> + compare_get_sec_indev, >>> compare_set_sec_indev, >>> + NULL); >>> + object_property_add_str(obj, "outdev", >>> + compare_get_outdev, compare_set_outdev, >>> + NULL); >>> +} >>> + >>> +static void colo_compare_finalize(Object *obj) >>> +{ >>> + CompareState *s = COLO_COMPARE(obj); >>> + >>> + if (s->chr_pri_in) { >>> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >>> + qemu_chr_fe_release(s->chr_pri_in); >>> + } >>> + if (s->chr_sec_in) { >>> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >>> + qemu_chr_fe_release(s->chr_sec_in); >>> + } >>> + if (s->chr_out) { >>> + qemu_chr_fe_release(s->chr_out); >>> + } >>> + >>> + g_free(s->pri_indev); >>> + g_free(s->sec_indev); >>> + g_free(s->outdev); >>> +} >>> + >>> +static const TypeInfo colo_compare_info = { >>> + .name = TYPE_COLO_COMPARE, >>> + .parent = TYPE_OBJECT, >>> + .instance_size = sizeof(CompareState), >>> + .instance_init = colo_compare_init, >>> + .instance_finalize = colo_compare_finalize, >>> + .class_size = sizeof(CompareClass), >>> + .class_init = colo_compare_class_init, >>> + .interfaces = (InterfaceInfo[]) { >>> + { TYPE_USER_CREATABLE }, >>> + { } >>> + } >>> +}; >>> + >>> +static void register_types(void) >>> +{ >>> + type_register_static(&colo_compare_info); >>> +} >>> + >>> +type_init(register_types); >>> diff --git a/qemu-options.hx b/qemu-options.hx >>> index 587de8f..33d5d0b 100644 >>> --- a/qemu-options.hx >>> +++ b/qemu-options.hx >>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} >>> to the file specified by >>> The file format is libpcap, so it can be analyzed with tools such >>> as tcpdump >>> or Wireshark. >>> +@item -object >>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >>> +outdev=@var{chardevid} >>> + >>> +Colo-compare gets packet from primary_in@var{chardevid} and >>> secondary_in@var{chardevid}, than compare primary packet with >>> +secondary packet. If the packet same, we will output primary >> >> s/If the packet same/If the packets are same/. > > OK. > >> >>> +packet to outdev@var{chardevid}, else we will notify colo-frame >>> +do checkpoint and send primary packet to outdev@var{chardevid}. >>> + >>> +we can use it with the help of filter-mirror and filter-redirector. >> >> s/we/We/ and looks like colo compare must be used with the help of >> mirror and redirector? > > Currently yes. Then please change the doc here. Thanks > >> >>> + >>> +@example >>> + >>> +primary: >>> +-netdev >>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>> +-object >>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>> +-object >>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>> +-object >>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>> + >>> +secondary: >>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>> script=/etc/qemu-ifdown >>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >>> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>> + >>> +@end example >>> + >>> +If you want to know the detail of above command line, you can read >>> +the colo-compare git log. >>> + >>> @item -object >>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>> @item -object >>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>> diff --git a/vl.c b/vl.c >>> index cbe51ac..c6b9a6f 100644 >>> --- a/vl.c >>> +++ b/vl.c >>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >>> *type) >>> if (g_str_equal(type, "filter-buffer") || >>> g_str_equal(type, "filter-dump") || >>> g_str_equal(type, "filter-mirror") || >>> - g_str_equal(type, "filter-redirector")) { >>> + g_str_equal(type, "filter-redirector") || >>> + g_str_equal(type, "colo-compare")) { >>> return false; >>> } >> >> >> >> . >> >
On 08/31/2016 05:20 PM, Jason Wang wrote: > > > On 2016年08月31日 17:03, Zhang Chen wrote: >> >> >> On 08/31/2016 03:53 PM, Jason Wang wrote: >>> >>> >>> On 2016年08月17日 16:10, Zhang Chen wrote: >>>> This a COLO net ascii figure: >>>> >>>> Primary qemu Secondary qemu >>>> +--------------------------------------------------------------+ >>>> +----------------------------------------------------------------+ >>>> | +----------------------------------------------------------+ >>>> | | >>>> +-----------------------------------------------------------+ | >>>> | | | | | | | | >>>> | | guest | | | | >>>> guest | | >>>> | | | | | | | | >>>> | +-------^--------------------------+-----------------------+ >>>> | | >>>> +---------------------+--------+----------------------------+ | >>>> | | | | | ^ >>>> | | >>>> | | | | | | >>>> | | >>>> | | >>>> +------------------------------------------------------+ >>>> | | | | >>>> |netfilter| | | | | | >>>> netfilter | | | >>>> | +----------+ +----------------------------+ | | | >>>> +-----------------------------------------------------------+ | >>>> | | | | | | out | | | >>>> | | | filter excute order | | >>>> | | | | +-----------------------------+ | | | >>>> | | | +-------------------> | | >>>> | | | | | | | | | | | >>>> | | | TCP | | >>>> | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| >>>> | | | +------------+ +---+----+---v+rewriter++ +------------+ | | >>>> | | | | | | | | |in | |in | | | | >>>> | | | | | | | | | >>>> | | | filter | | filter | | filter +------> colo <------+ >>>> +--------> filter +--> adjust | adjust +--> filter | | | >>>> | | | mirror | |redirector| |redirector| | | compare | | >>>> | | | | redirector | | ack | seq | | redirector | >>>> | | >>>> | | | | | | | | | | | | >>>> | | | | | | | | | | | | >>>> | | +----^-----+ +----+-----+ +----------+ | +---------+ | >>>> | | | +------------+ +--------+--------------+ +---+--------+ | | >>>> | | | tx | rx rx | | | | >>>> | tx all | rx | | >>>> | | | | | | | | >>>> +-----------------------------------------------------------+ | >>>> | | | +--------------+ | | | | | | >>>> | | | filter excute order | | | | | | | >>>> | | | +----------------> | | | >>>> +--------------------------------------------------------+ | >>>> | +-----------------------------------------+ | | | >>>> | | | | | | >>>> +--------------------------------------------------------------+ >>>> +----------------------------------------------------------------+ >>>> |guest receive | guest send >>>> | | >>>> +--------+----------------------------v------------------------+ >>>> | | NOTE: filter direction is rx/tx/all >>>> | tap | rx:receive packets sent to the netdev >>>> | | tx:receive packets sent by the netdev >>>> +--------------------------------------------------------------+ >>> >>> It's better to add a doc under docs to explain this configuration in >>> detail on top of this series. >>> >> >> As you say, Am I add /docs/colo-proxy.txt to explain it or add this >> in hailiang's COLO-FT.txt after merge? Can you give me a way for doc? >> >>>> In COLO-compare, we do packet comparing job. >>>> Packets coming from the primary char indev will be sent to outdev. >>>> Packets coming from the secondary char dev will be dropped after >>>> comparing. >>>> colo-comapre need two input chardev and one output chardev: >>>> primary_in=chardev1-id (source: primary send packet) >>>> secondary_in=chardev2-id (source: secondary send packet) >>>> outdev=chardev3-id >>>> >>>> usage: >>>> >>>> primary: >>>> -netdev >>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>>> -object >>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>>> -object >>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>>> -object >>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>>> >>>> secondary: >>>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>>> script=/etc/qemu-ifdown >>>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>>> -chardev socket,id=red0,host=3.3.3.3,port=9003 >>>> -chardev socket,id=red1,host=3.3.3.3,port=9004 >>>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>>> >>>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >>>> --- >>>> net/Makefile.objs | 1 + >>>> net/colo-compare.c | 284 >>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> qemu-options.hx | 39 ++++++++ >>>> vl.c | 3 +- >>>> 4 files changed, 326 insertions(+), 1 deletion(-) >>>> create mode 100644 net/colo-compare.c >>>> >>>> diff --git a/net/Makefile.objs b/net/Makefile.objs >>>> index b7c22fd..ba92f73 100644 >>>> --- a/net/Makefile.objs >>>> +++ b/net/Makefile.objs >>>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >>>> common-obj-y += filter.o >>>> common-obj-y += filter-buffer.o >>>> common-obj-y += filter-mirror.o >>>> +common-obj-y += colo-compare.o >>>> diff --git a/net/colo-compare.c b/net/colo-compare.c >>>> new file mode 100644 >>>> index 0000000..cdc3e0e >>>> --- /dev/null >>>> +++ b/net/colo-compare.c >>>> @@ -0,0 +1,284 @@ >>>> +/* >>>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop >>>> Service (COLO) >>>> + * (a.k.a. Fault Tolerance or Continuous Replication) >>>> + * >>>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >>>> + * Copyright (c) 2016 FUJITSU LIMITED >>>> + * Copyright (c) 2016 Intel Corporation >>>> + * >>>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>>> + * >>>> + * This work is licensed under the terms of the GNU GPL, version 2 or >>>> + * later. See the COPYING file in the top-level directory. >>>> + */ >>>> + >>>> +#include "qemu/osdep.h" >>>> +#include "qemu/error-report.h" >>>> +#include "qemu-common.h" >>>> +#include "qapi/qmp/qerror.h" >>>> +#include "qapi/error.h" >>>> +#include "net/net.h" >>>> +#include "net/vhost_net.h" >>> >>> Looks unnecessary. >> >> I will remove it. >> >>> >>>> +#include "qom/object_interfaces.h" >>>> +#include "qemu/iov.h" >>>> +#include "qom/object.h" >>>> +#include "qemu/typedefs.h" >>>> +#include "net/queue.h" >>>> +#include "sysemu/char.h" >>>> +#include "qemu/sockets.h" >>>> +#include "qapi-visit.h" >>>> + >>>> +#define TYPE_COLO_COMPARE "colo-compare" >>>> +#define COLO_COMPARE(obj) \ >>>> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >>>> + >>>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >>>> + >>>> +typedef struct CompareState { >>>> + Object parent; >>>> + >>>> + char *pri_indev; >>>> + char *sec_indev; >>>> + char *outdev; >>>> + CharDriverState *chr_pri_in; >>>> + CharDriverState *chr_sec_in; >>>> + CharDriverState *chr_out; >>>> + QTAILQ_ENTRY(CompareState) next; >>> >>> This looks not used in this series but in commit "colo-compare and >>> filter-rewriter work with colo-frame". We'd better delay the >>> introducing to that patch. >> >> OK~ I got your point. >> >>> >>>> + SocketReadState pri_rs; >>>> + SocketReadState sec_rs; >>>> +} CompareState; >>>> + >>>> +typedef struct CompareClass { >>>> + ObjectClass parent_class; >>>> +} CompareClass; >>>> + >>>> +typedef struct CompareChardevProps { >>>> + bool is_socket; >>>> + bool is_unix; >>>> +} CompareChardevProps; >>>> + >>>> +static char *compare_get_pri_indev(Object *obj, Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + return g_strdup(s->pri_indev); >>>> +} >>>> + >>>> +static void compare_set_pri_indev(Object *obj, const char *value, >>>> Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + g_free(s->pri_indev); >>>> + s->pri_indev = g_strdup(value); >>>> +} >>>> + >>>> +static char *compare_get_sec_indev(Object *obj, Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + return g_strdup(s->sec_indev); >>>> +} >>>> + >>>> +static void compare_set_sec_indev(Object *obj, const char *value, >>>> Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + g_free(s->sec_indev); >>>> + s->sec_indev = g_strdup(value); >>>> +} >>>> + >>>> +static char *compare_get_outdev(Object *obj, Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + return g_strdup(s->outdev); >>>> +} >>>> + >>>> +static void compare_set_outdev(Object *obj, const char *value, >>>> Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + g_free(s->outdev); >>>> + s->outdev = g_strdup(value); >>>> +} >>>> + >>>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >>>> +{ >>>> + /* if packet_enqueue pri pkt failed we will send unsupported >>>> packet */ >>>> +} >>>> + >>>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >>>> +{ >>>> + /* if packet_enqueue sec pkt failed we will notify trace */ >>>> +} >>>> + >>>> +static int compare_chardev_opts(void *opaque, >>>> + const char *name, const char *value, >>>> + Error **errp) >>>> +{ >>>> + CompareChardevProps *props = opaque; >>>> + >>>> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == >>>> 0) { >>>> + props->is_socket = true; >>>> + } else if (strcmp(name, "host") == 0) { >>> >>> Typo? net_vhost_chardev_opts() did: >>> >>> } else if (strcmp(name, "path") == 0) { >>> props->is_unix = true; >>> } >>> >>> >> >> No, In colo-compare we use chardev like this: >> >> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> >> If we only use "path" here will trigger a error. >> Should I add anthor "path" here? > > If I understand the code correctly, "is_unix" means "is unix domain > socket"? If yes, according to the help: > > -chardev > socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds] > [,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off] > [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp) > -chardev > socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds] > [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix) > > "host" will not be used for UNIX domain socket. > > And if UNIX domain socket is not supported, there's probably no need > to differentiate it from other types. OK, I will remove the "is_unix" in next version. > >> >> >>> >>>> + props->is_unix = true; >>>> + } else if (strcmp(name, "port") == 0) { >>>> + } else if (strcmp(name, "server") == 0) { >>>> + } else if (strcmp(name, "wait") == 0) { >>>> + } else { >>>> + error_setg(errp, >>>> + "COLO-compare does not support a chardev with >>>> option %s=%s", >>>> + name, value); >>>> + return -1; >>>> + } >>>> + return 0; >>>> +} >>>> + >>>> +/* >>>> + * called from the main thread on the primary >>>> + * to setup colo-compare. >>>> + */ >>>> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(uc); >>>> + CompareChardevProps props; >>>> + >>>> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >>>> + error_setg(errp, "colo compare needs 'primary_in' ," >>>> + "'secondary_in','outdev' property set"); >>>> + return; >>>> + } else if (!strcmp(s->pri_indev, s->outdev) || >>>> + !strcmp(s->sec_indev, s->outdev) || >>>> + !strcmp(s->pri_indev, s->sec_indev)) { >>>> + error_setg(errp, "'indev' and 'outdev' could not be same " >>>> + "for compare module"); >>>> + return; >>>> + } >>>> + >>>> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >>>> + if (s->chr_pri_in == NULL) { >>>> + error_setg(errp, "Primary IN Device '%s' not found", >>>> + s->pri_indev); >>>> + return; >>>> + } >>>> + >>>> + /* inspect chardev opts */ >>>> + memset(&props, 0, sizeof(props)); >>>> + if (qemu_opt_foreach(s->chr_pri_in->opts, >>>> compare_chardev_opts, &props, errp)) { >>>> + return; >>>> + } >>>> + >>>> + if (!props.is_socket || !props.is_unix) { >>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>> + s->pri_indev); >>>> + return; >>>> + } >>>> + >>>> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >>>> + if (s->chr_sec_in == NULL) { >>>> + error_setg(errp, "Secondary IN Device '%s' not found", >>>> + s->sec_indev); >>>> + return; >>>> + } >>>> + >>>> + memset(&props, 0, sizeof(props)); >>>> + if (qemu_opt_foreach(s->chr_sec_in->opts, >>>> compare_chardev_opts, &props, errp)) { >>>> + return; >>>> + } >>>> + >>>> + if (!props.is_socket || !props.is_unix) { >>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>> + s->sec_indev); >>> >>> I believe tcp socket is also supported? >> >> If I understand correctly, "tcp socket" in here is the "-chardev >> socket". >> I will rename "unix socket" to "tcp socket". >> >>> >>>> + return; >>>> + } >>>> + >>>> + s->chr_out = qemu_chr_find(s->outdev); >>>> + if (s->chr_out == NULL) { >>>> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >>>> + return; >>>> + } >>>> + >>>> + memset(&props, 0, sizeof(props)); >>>> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, >>>> &props, errp)) { >>>> + return; >>>> + } >>>> + >>>> + if (!props.is_socket || !props.is_unix) { >>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>> + s->outdev); >>> >>> Ditto, and there's code duplication, please introduce a helper to do >>> above. >> >> I don't understand what the "helper"? >> In here we check each chardev, will I change to "goto error;" ? > > A helper to avoid the code duplication for socket type inspection for > pri_in,scr_in and chr_out. I got it~~ I will add it in next version. > >> >>> >>>> + return; >>>> + } >>>> + >>>> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >>>> + >>>> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >>>> + >>>> + qemu_chr_fe_claim_no_fail(s->chr_out); >>>> + >>>> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >>>> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >>>> + >>>> + return; >>>> +} >>>> + >>>> +static void colo_compare_class_init(ObjectClass *oc, void *data) >>>> +{ >>>> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >>>> + >>>> + ucc->complete = colo_compare_complete; >>>> +} >>>> + >>>> +static void colo_compare_init(Object *obj) >>>> +{ >>>> + object_property_add_str(obj, "primary_in", >>>> + compare_get_pri_indev, >>>> compare_set_pri_indev, >>>> + NULL); >>>> + object_property_add_str(obj, "secondary_in", >>>> + compare_get_sec_indev, >>>> compare_set_sec_indev, >>>> + NULL); >>>> + object_property_add_str(obj, "outdev", >>>> + compare_get_outdev, compare_set_outdev, >>>> + NULL); >>>> +} >>>> + >>>> +static void colo_compare_finalize(Object *obj) >>>> +{ >>>> + CompareState *s = COLO_COMPARE(obj); >>>> + >>>> + if (s->chr_pri_in) { >>>> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >>>> + qemu_chr_fe_release(s->chr_pri_in); >>>> + } >>>> + if (s->chr_sec_in) { >>>> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >>>> + qemu_chr_fe_release(s->chr_sec_in); >>>> + } >>>> + if (s->chr_out) { >>>> + qemu_chr_fe_release(s->chr_out); >>>> + } >>>> + >>>> + g_free(s->pri_indev); >>>> + g_free(s->sec_indev); >>>> + g_free(s->outdev); >>>> +} >>>> + >>>> +static const TypeInfo colo_compare_info = { >>>> + .name = TYPE_COLO_COMPARE, >>>> + .parent = TYPE_OBJECT, >>>> + .instance_size = sizeof(CompareState), >>>> + .instance_init = colo_compare_init, >>>> + .instance_finalize = colo_compare_finalize, >>>> + .class_size = sizeof(CompareClass), >>>> + .class_init = colo_compare_class_init, >>>> + .interfaces = (InterfaceInfo[]) { >>>> + { TYPE_USER_CREATABLE }, >>>> + { } >>>> + } >>>> +}; >>>> + >>>> +static void register_types(void) >>>> +{ >>>> + type_register_static(&colo_compare_info); >>>> +} >>>> + >>>> +type_init(register_types); >>>> diff --git a/qemu-options.hx b/qemu-options.hx >>>> index 587de8f..33d5d0b 100644 >>>> --- a/qemu-options.hx >>>> +++ b/qemu-options.hx >>>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} >>>> to the file specified by >>>> The file format is libpcap, so it can be analyzed with tools such >>>> as tcpdump >>>> or Wireshark. >>>> +@item -object >>>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >>>> +outdev=@var{chardevid} >>>> + >>>> +Colo-compare gets packet from primary_in@var{chardevid} and >>>> secondary_in@var{chardevid}, than compare primary packet with >>>> +secondary packet. If the packet same, we will output primary >>> >>> s/If the packet same/If the packets are same/. >> >> OK. >> >>> >>>> +packet to outdev@var{chardevid}, else we will notify colo-frame >>>> +do checkpoint and send primary packet to outdev@var{chardevid}. >>>> + >>>> +we can use it with the help of filter-mirror and filter-redirector. >>> >>> s/we/We/ and looks like colo compare must be used with the help of >>> mirror and redirector? >> >> Currently yes. > > Then please change the doc here. s/We can use it/We must use it. Thanks Zhang Chen > > Thanks > >> >>> >>>> + >>>> +@example >>>> + >>>> +primary: >>>> +-netdev >>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>>> +-object >>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>>> +-object >>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>>> +-object >>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>>> + >>>> +secondary: >>>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>>> script=/etc/qemu-ifdown >>>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>>> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >>>> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >>>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>>> + >>>> +@end example >>>> + >>>> +If you want to know the detail of above command line, you can read >>>> +the colo-compare git log. >>>> + >>>> @item -object >>>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>>> @item -object >>>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>>> diff --git a/vl.c b/vl.c >>>> index cbe51ac..c6b9a6f 100644 >>>> --- a/vl.c >>>> +++ b/vl.c >>>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >>>> *type) >>>> if (g_str_equal(type, "filter-buffer") || >>>> g_str_equal(type, "filter-dump") || >>>> g_str_equal(type, "filter-mirror") || >>>> - g_str_equal(type, "filter-redirector")) { >>>> + g_str_equal(type, "filter-redirector") || >>>> + g_str_equal(type, "colo-compare")) { >>>> return false; >>>> } >>> >>> >>> >>> . >>> >> > > > > . >
On 08/31/2016 05:39 PM, Zhang Chen wrote: > > > On 08/31/2016 05:20 PM, Jason Wang wrote: >> >> >> On 2016年08月31日 17:03, Zhang Chen wrote: >>> >>> >>> On 08/31/2016 03:53 PM, Jason Wang wrote: >>>> >>>> >>>> On 2016年08月17日 16:10, Zhang Chen wrote: >>>>> This a COLO net ascii figure: >>>>> >>>>> Primary qemu Secondary qemu >>>>> +--------------------------------------------------------------+ >>>>> +----------------------------------------------------------------+ >>>>> | +----------------------------------------------------------+ >>>>> | | >>>>> +-----------------------------------------------------------+ | >>>>> | | | | | | | | >>>>> | | guest | | | | >>>>> guest | | >>>>> | | | | | | | | >>>>> | +-------^--------------------------+-----------------------+ >>>>> | | >>>>> +---------------------+--------+----------------------------+ | >>>>> | | | | | >>>>> ^ | | >>>>> | | | | | >>>>> | | | >>>>> | | >>>>> +------------------------------------------------------+ >>>>> | | | | >>>>> |netfilter| | | | | | >>>>> netfilter | | | >>>>> | +----------+ +----------------------------+ | | | >>>>> +-----------------------------------------------------------+ | >>>>> | | | | | | out | | | >>>>> | | | filter excute order | | >>>>> | | | | +-----------------------------+ | | | >>>>> | | | +-------------------> | | >>>>> | | | | | | | | | | | >>>>> | | | TCP | | >>>>> | | +-----+--+-+ +-----v----+ +-----v----+ |pri >>>>> +----+----+sec| | | | +------------+ >>>>> +---+----+---v+rewriter++ +------------+ | | >>>>> | | | | | | | | |in | |in | | | >>>>> | | | | | | | | | | >>>>> | | | filter | | filter | | filter +------> colo <------+ >>>>> +--------> filter +--> adjust | adjust +--> filter | | | >>>>> | | | mirror | |redirector| |redirector| | | compare | | >>>>> | | | | redirector | | ack | seq | | redirector | >>>>> | | >>>>> | | | | | | | | | | | | | | | >>>>> | | | | | | | | | >>>>> | | +----^-----+ +----+-----+ +----------+ | +---------+ | | >>>>> | | +------------+ +--------+--------------+ +---+--------+ | | >>>>> | | | tx | rx rx | | | | >>>>> | tx all | rx | | >>>>> | | | | | | | | >>>>> +-----------------------------------------------------------+ | >>>>> | | | +--------------+ | | | | >>>>> | | >>>>> | | | filter excute order | | | | | >>>>> | | >>>>> | | | +----------------> | | | >>>>> +--------------------------------------------------------+ | >>>>> | +-----------------------------------------+ | | | >>>>> | | | | | | >>>>> +--------------------------------------------------------------+ >>>>> +----------------------------------------------------------------+ >>>>> |guest receive | guest send >>>>> | | >>>>> +--------+----------------------------v------------------------+ >>>>> | | NOTE: filter direction is rx/tx/all >>>>> | tap | rx:receive packets sent to the netdev >>>>> | | tx:receive packets sent by the netdev >>>>> +--------------------------------------------------------------+ >>>> >>>> It's better to add a doc under docs to explain this configuration >>>> in detail on top of this series. >>>> >>> >>> As you say, Am I add /docs/colo-proxy.txt to explain it or add this >>> in hailiang's COLO-FT.txt after merge? > > Can you give me a way for doc? > > Sorry, I misunderstand hailiang's mean, I will add /docs/colo-proxy.txt in next version. Thanks Zhang Chen >>> >>>>> In COLO-compare, we do packet comparing job. >>>>> Packets coming from the primary char indev will be sent to outdev. >>>>> Packets coming from the secondary char dev will be dropped after >>>>> comparing. >>>>> colo-comapre need two input chardev and one output chardev: >>>>> primary_in=chardev1-id (source: primary send packet) >>>>> secondary_in=chardev2-id (source: secondary send packet) >>>>> outdev=chardev3-id >>>>> >>>>> usage: >>>>> >>>>> primary: >>>>> -netdev >>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>>>> >>>>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>>>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>>>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>>>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>>>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>>>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>>>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>>>> -object >>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>>>> -object >>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>>>> -object >>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>>>> >>>>> secondary: >>>>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>>>> script=/etc/qemu-ifdown >>>>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>>>> -chardev socket,id=red0,host=3.3.3.3,port=9003 >>>>> -chardev socket,id=red1,host=3.3.3.3,port=9004 >>>>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>>>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>>>> >>>>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>>>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> >>>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> >>>>> --- >>>>> net/Makefile.objs | 1 + >>>>> net/colo-compare.c | 284 >>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>> qemu-options.hx | 39 ++++++++ >>>>> vl.c | 3 +- >>>>> 4 files changed, 326 insertions(+), 1 deletion(-) >>>>> create mode 100644 net/colo-compare.c >>>>> >>>>> diff --git a/net/Makefile.objs b/net/Makefile.objs >>>>> index b7c22fd..ba92f73 100644 >>>>> --- a/net/Makefile.objs >>>>> +++ b/net/Makefile.objs >>>>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >>>>> common-obj-y += filter.o >>>>> common-obj-y += filter-buffer.o >>>>> common-obj-y += filter-mirror.o >>>>> +common-obj-y += colo-compare.o >>>>> diff --git a/net/colo-compare.c b/net/colo-compare.c >>>>> new file mode 100644 >>>>> index 0000000..cdc3e0e >>>>> --- /dev/null >>>>> +++ b/net/colo-compare.c >>>>> @@ -0,0 +1,284 @@ >>>>> +/* >>>>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop >>>>> Service (COLO) >>>>> + * (a.k.a. Fault Tolerance or Continuous Replication) >>>>> + * >>>>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >>>>> + * Copyright (c) 2016 FUJITSU LIMITED >>>>> + * Copyright (c) 2016 Intel Corporation >>>>> + * >>>>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> >>>>> + * >>>>> + * This work is licensed under the terms of the GNU GPL, version >>>>> 2 or >>>>> + * later. See the COPYING file in the top-level directory. >>>>> + */ >>>>> + >>>>> +#include "qemu/osdep.h" >>>>> +#include "qemu/error-report.h" >>>>> +#include "qemu-common.h" >>>>> +#include "qapi/qmp/qerror.h" >>>>> +#include "qapi/error.h" >>>>> +#include "net/net.h" >>>>> +#include "net/vhost_net.h" >>>> >>>> Looks unnecessary. >>> >>> I will remove it. >>> >>>> >>>>> +#include "qom/object_interfaces.h" >>>>> +#include "qemu/iov.h" >>>>> +#include "qom/object.h" >>>>> +#include "qemu/typedefs.h" >>>>> +#include "net/queue.h" >>>>> +#include "sysemu/char.h" >>>>> +#include "qemu/sockets.h" >>>>> +#include "qapi-visit.h" >>>>> + >>>>> +#define TYPE_COLO_COMPARE "colo-compare" >>>>> +#define COLO_COMPARE(obj) \ >>>>> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >>>>> + >>>>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >>>>> + >>>>> +typedef struct CompareState { >>>>> + Object parent; >>>>> + >>>>> + char *pri_indev; >>>>> + char *sec_indev; >>>>> + char *outdev; >>>>> + CharDriverState *chr_pri_in; >>>>> + CharDriverState *chr_sec_in; >>>>> + CharDriverState *chr_out; >>>>> + QTAILQ_ENTRY(CompareState) next; >>>> >>>> This looks not used in this series but in commit "colo-compare and >>>> filter-rewriter work with colo-frame". We'd better delay the >>>> introducing to that patch. >>> >>> OK~ I got your point. >>> >>>> >>>>> + SocketReadState pri_rs; >>>>> + SocketReadState sec_rs; >>>>> +} CompareState; >>>>> + >>>>> +typedef struct CompareClass { >>>>> + ObjectClass parent_class; >>>>> +} CompareClass; >>>>> + >>>>> +typedef struct CompareChardevProps { >>>>> + bool is_socket; >>>>> + bool is_unix; >>>>> +} CompareChardevProps; >>>>> + >>>>> +static char *compare_get_pri_indev(Object *obj, Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + return g_strdup(s->pri_indev); >>>>> +} >>>>> + >>>>> +static void compare_set_pri_indev(Object *obj, const char *value, >>>>> Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + g_free(s->pri_indev); >>>>> + s->pri_indev = g_strdup(value); >>>>> +} >>>>> + >>>>> +static char *compare_get_sec_indev(Object *obj, Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + return g_strdup(s->sec_indev); >>>>> +} >>>>> + >>>>> +static void compare_set_sec_indev(Object *obj, const char *value, >>>>> Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + g_free(s->sec_indev); >>>>> + s->sec_indev = g_strdup(value); >>>>> +} >>>>> + >>>>> +static char *compare_get_outdev(Object *obj, Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + return g_strdup(s->outdev); >>>>> +} >>>>> + >>>>> +static void compare_set_outdev(Object *obj, const char *value, >>>>> Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + g_free(s->outdev); >>>>> + s->outdev = g_strdup(value); >>>>> +} >>>>> + >>>>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >>>>> +{ >>>>> + /* if packet_enqueue pri pkt failed we will send unsupported >>>>> packet */ >>>>> +} >>>>> + >>>>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >>>>> +{ >>>>> + /* if packet_enqueue sec pkt failed we will notify trace */ >>>>> +} >>>>> + >>>>> +static int compare_chardev_opts(void *opaque, >>>>> + const char *name, const char *value, >>>>> + Error **errp) >>>>> +{ >>>>> + CompareChardevProps *props = opaque; >>>>> + >>>>> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") >>>>> == 0) { >>>>> + props->is_socket = true; >>>>> + } else if (strcmp(name, "host") == 0) { >>>> >>>> Typo? net_vhost_chardev_opts() did: >>>> >>>> } else if (strcmp(name, "path") == 0) { >>>> props->is_unix = true; >>>> } >>>> >>>> >>> >>> No, In colo-compare we use chardev like this: >>> >>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>> >>> If we only use "path" here will trigger a error. >>> Should I add anthor "path" here? >> >> If I understand the code correctly, "is_unix" means "is unix domain >> socket"? If yes, according to the help: >> >> -chardev >> socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds] >> [,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off] >> [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp) >> -chardev >> socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds] >> [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix) >> >> "host" will not be used for UNIX domain socket. >> >> And if UNIX domain socket is not supported, there's probably no need >> to differentiate it from other types. > > OK, I will remove the "is_unix" in next version. > > >> >>> >>> >>>> >>>>> + props->is_unix = true; >>>>> + } else if (strcmp(name, "port") == 0) { >>>>> + } else if (strcmp(name, "server") == 0) { >>>>> + } else if (strcmp(name, "wait") == 0) { >>>>> + } else { >>>>> + error_setg(errp, >>>>> + "COLO-compare does not support a chardev with >>>>> option %s=%s", >>>>> + name, value); >>>>> + return -1; >>>>> + } >>>>> + return 0; >>>>> +} >>>>> + >>>>> +/* >>>>> + * called from the main thread on the primary >>>>> + * to setup colo-compare. >>>>> + */ >>>>> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(uc); >>>>> + CompareChardevProps props; >>>>> + >>>>> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >>>>> + error_setg(errp, "colo compare needs 'primary_in' ," >>>>> + "'secondary_in','outdev' property set"); >>>>> + return; >>>>> + } else if (!strcmp(s->pri_indev, s->outdev) || >>>>> + !strcmp(s->sec_indev, s->outdev) || >>>>> + !strcmp(s->pri_indev, s->sec_indev)) { >>>>> + error_setg(errp, "'indev' and 'outdev' could not be same " >>>>> + "for compare module"); >>>>> + return; >>>>> + } >>>>> + >>>>> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >>>>> + if (s->chr_pri_in == NULL) { >>>>> + error_setg(errp, "Primary IN Device '%s' not found", >>>>> + s->pri_indev); >>>>> + return; >>>>> + } >>>>> + >>>>> + /* inspect chardev opts */ >>>>> + memset(&props, 0, sizeof(props)); >>>>> + if (qemu_opt_foreach(s->chr_pri_in->opts, >>>>> compare_chardev_opts, &props, errp)) { >>>>> + return; >>>>> + } >>>>> + >>>>> + if (!props.is_socket || !props.is_unix) { >>>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>>> + s->pri_indev); >>>>> + return; >>>>> + } >>>>> + >>>>> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >>>>> + if (s->chr_sec_in == NULL) { >>>>> + error_setg(errp, "Secondary IN Device '%s' not found", >>>>> + s->sec_indev); >>>>> + return; >>>>> + } >>>>> + >>>>> + memset(&props, 0, sizeof(props)); >>>>> + if (qemu_opt_foreach(s->chr_sec_in->opts, >>>>> compare_chardev_opts, &props, errp)) { >>>>> + return; >>>>> + } >>>>> + >>>>> + if (!props.is_socket || !props.is_unix) { >>>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>>> + s->sec_indev); >>>> >>>> I believe tcp socket is also supported? >>> >>> If I understand correctly, "tcp socket" in here is the "-chardev >>> socket". >>> I will rename "unix socket" to "tcp socket". >>> >>>> >>>>> + return; >>>>> + } >>>>> + >>>>> + s->chr_out = qemu_chr_find(s->outdev); >>>>> + if (s->chr_out == NULL) { >>>>> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >>>>> + return; >>>>> + } >>>>> + >>>>> + memset(&props, 0, sizeof(props)); >>>>> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, >>>>> &props, errp)) { >>>>> + return; >>>>> + } >>>>> + >>>>> + if (!props.is_socket || !props.is_unix) { >>>>> + error_setg(errp, "chardev \"%s\" is not a unix socket", >>>>> + s->outdev); >>>> >>>> Ditto, and there's code duplication, please introduce a helper to >>>> do above. >>> >>> I don't understand what the "helper"? >>> In here we check each chardev, will I change to "goto error;" ? >> >> A helper to avoid the code duplication for socket type inspection for >> pri_in,scr_in and chr_out. > > I got it~~ > I will add it in next version. > > >> >>> >>>> >>>>> + return; >>>>> + } >>>>> + >>>>> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >>>>> + >>>>> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >>>>> + >>>>> + qemu_chr_fe_claim_no_fail(s->chr_out); >>>>> + >>>>> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >>>>> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >>>>> + >>>>> + return; >>>>> +} >>>>> + >>>>> +static void colo_compare_class_init(ObjectClass *oc, void *data) >>>>> +{ >>>>> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >>>>> + >>>>> + ucc->complete = colo_compare_complete; >>>>> +} >>>>> + >>>>> +static void colo_compare_init(Object *obj) >>>>> +{ >>>>> + object_property_add_str(obj, "primary_in", >>>>> + compare_get_pri_indev, >>>>> compare_set_pri_indev, >>>>> + NULL); >>>>> + object_property_add_str(obj, "secondary_in", >>>>> + compare_get_sec_indev, >>>>> compare_set_sec_indev, >>>>> + NULL); >>>>> + object_property_add_str(obj, "outdev", >>>>> + compare_get_outdev, compare_set_outdev, >>>>> + NULL); >>>>> +} >>>>> + >>>>> +static void colo_compare_finalize(Object *obj) >>>>> +{ >>>>> + CompareState *s = COLO_COMPARE(obj); >>>>> + >>>>> + if (s->chr_pri_in) { >>>>> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, >>>>> NULL); >>>>> + qemu_chr_fe_release(s->chr_pri_in); >>>>> + } >>>>> + if (s->chr_sec_in) { >>>>> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, >>>>> NULL); >>>>> + qemu_chr_fe_release(s->chr_sec_in); >>>>> + } >>>>> + if (s->chr_out) { >>>>> + qemu_chr_fe_release(s->chr_out); >>>>> + } >>>>> + >>>>> + g_free(s->pri_indev); >>>>> + g_free(s->sec_indev); >>>>> + g_free(s->outdev); >>>>> +} >>>>> + >>>>> +static const TypeInfo colo_compare_info = { >>>>> + .name = TYPE_COLO_COMPARE, >>>>> + .parent = TYPE_OBJECT, >>>>> + .instance_size = sizeof(CompareState), >>>>> + .instance_init = colo_compare_init, >>>>> + .instance_finalize = colo_compare_finalize, >>>>> + .class_size = sizeof(CompareClass), >>>>> + .class_init = colo_compare_class_init, >>>>> + .interfaces = (InterfaceInfo[]) { >>>>> + { TYPE_USER_CREATABLE }, >>>>> + { } >>>>> + } >>>>> +}; >>>>> + >>>>> +static void register_types(void) >>>>> +{ >>>>> + type_register_static(&colo_compare_info); >>>>> +} >>>>> + >>>>> +type_init(register_types); >>>>> diff --git a/qemu-options.hx b/qemu-options.hx >>>>> index 587de8f..33d5d0b 100644 >>>>> --- a/qemu-options.hx >>>>> +++ b/qemu-options.hx >>>>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev >>>>> @var{dev} to the file specified by >>>>> The file format is libpcap, so it can be analyzed with tools >>>>> such as tcpdump >>>>> or Wireshark. >>>>> +@item -object >>>>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >>>>> +outdev=@var{chardevid} >>>>> + >>>>> +Colo-compare gets packet from primary_in@var{chardevid} and >>>>> secondary_in@var{chardevid}, than compare primary packet with >>>>> +secondary packet. If the packet same, we will output primary >>>> >>>> s/If the packet same/If the packets are same/. >>> >>> OK. >>> >>>> >>>>> +packet to outdev@var{chardevid}, else we will notify colo-frame >>>>> +do checkpoint and send primary packet to outdev@var{chardevid}. >>>>> + >>>>> +we can use it with the help of filter-mirror and filter-redirector. >>>> >>>> s/we/We/ and looks like colo compare must be used with the help of >>>> mirror and redirector? >>> >>> Currently yes. >> >> Then please change the doc here. > > s/We can use it/We must use it. > > Thanks > Zhang Chen > >> >> Thanks >> >>> >>>> >>>>> + >>>>> +@example >>>>> + >>>>> +primary: >>>>> +-netdev >>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >>>>> >>>>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >>>>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >>>>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >>>>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >>>>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >>>>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >>>>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >>>>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >>>>> +-object >>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >>>>> +-object >>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >>>>> +-object >>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >>>>> + >>>>> +secondary: >>>>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >>>>> script=/etc/qemu-ifdown >>>>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >>>>> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >>>>> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >>>>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >>>>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >>>>> + >>>>> +@end example >>>>> + >>>>> +If you want to know the detail of above command line, you can read >>>>> +the colo-compare git log. >>>>> + >>>>> @item -object >>>>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>>>> @item -object >>>>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >>>>> diff --git a/vl.c b/vl.c >>>>> index cbe51ac..c6b9a6f 100644 >>>>> --- a/vl.c >>>>> +++ b/vl.c >>>>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >>>>> *type) >>>>> if (g_str_equal(type, "filter-buffer") || >>>>> g_str_equal(type, "filter-dump") || >>>>> g_str_equal(type, "filter-mirror") || >>>>> - g_str_equal(type, "filter-redirector")) { >>>>> + g_str_equal(type, "filter-redirector") || >>>>> + g_str_equal(type, "colo-compare")) { >>>>> return false; >>>>> } >>>> >>>> >>>> >>>> . >>>> >>> >> >> >> >> . >> >
diff --git a/net/Makefile.objs b/net/Makefile.objs index b7c22fd..ba92f73 100644 --- a/net/Makefile.objs +++ b/net/Makefile.objs @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o common-obj-y += filter.o common-obj-y += filter-buffer.o common-obj-y += filter-mirror.o +common-obj-y += colo-compare.o diff --git a/net/colo-compare.c b/net/colo-compare.c new file mode 100644 index 0000000..cdc3e0e --- /dev/null +++ b/net/colo-compare.c @@ -0,0 +1,284 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. + * Copyright (c) 2016 FUJITSU LIMITED + * Copyright (c) 2016 Intel Corporation + * + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qemu-common.h" +#include "qapi/qmp/qerror.h" +#include "qapi/error.h" +#include "net/net.h" +#include "net/vhost_net.h" +#include "qom/object_interfaces.h" +#include "qemu/iov.h" +#include "qom/object.h" +#include "qemu/typedefs.h" +#include "net/queue.h" +#include "sysemu/char.h" +#include "qemu/sockets.h" +#include "qapi-visit.h" + +#define TYPE_COLO_COMPARE "colo-compare" +#define COLO_COMPARE(obj) \ + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) + +#define COMPARE_READ_LEN_MAX NET_BUFSIZE + +typedef struct CompareState { + Object parent; + + char *pri_indev; + char *sec_indev; + char *outdev; + CharDriverState *chr_pri_in; + CharDriverState *chr_sec_in; + CharDriverState *chr_out; + QTAILQ_ENTRY(CompareState) next; + SocketReadState pri_rs; + SocketReadState sec_rs; +} CompareState; + +typedef struct CompareClass { + ObjectClass parent_class; +} CompareClass; + +typedef struct CompareChardevProps { + bool is_socket; + bool is_unix; +} CompareChardevProps; + +static char *compare_get_pri_indev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->pri_indev); +} + +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->pri_indev); + s->pri_indev = g_strdup(value); +} + +static char *compare_get_sec_indev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->sec_indev); +} + +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->sec_indev); + s->sec_indev = g_strdup(value); +} + +static char *compare_get_outdev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->outdev); +} + +static void compare_set_outdev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->outdev); + s->outdev = g_strdup(value); +} + +static void compare_pri_rs_finalize(SocketReadState *pri_rs) +{ + /* if packet_enqueue pri pkt failed we will send unsupported packet */ +} + +static void compare_sec_rs_finalize(SocketReadState *sec_rs) +{ + /* if packet_enqueue sec pkt failed we will notify trace */ +} + +static int compare_chardev_opts(void *opaque, + const char *name, const char *value, + Error **errp) +{ + CompareChardevProps *props = opaque; + + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { + props->is_socket = true; + } else if (strcmp(name, "host") == 0) { + props->is_unix = true; + } else if (strcmp(name, "port") == 0) { + } else if (strcmp(name, "server") == 0) { + } else if (strcmp(name, "wait") == 0) { + } else { + error_setg(errp, + "COLO-compare does not support a chardev with option %s=%s", + name, value); + return -1; + } + return 0; +} + +/* + * called from the main thread on the primary + * to setup colo-compare. + */ +static void colo_compare_complete(UserCreatable *uc, Error **errp) +{ + CompareState *s = COLO_COMPARE(uc); + CompareChardevProps props; + + if (!s->pri_indev || !s->sec_indev || !s->outdev) { + error_setg(errp, "colo compare needs 'primary_in' ," + "'secondary_in','outdev' property set"); + return; + } else if (!strcmp(s->pri_indev, s->outdev) || + !strcmp(s->sec_indev, s->outdev) || + !strcmp(s->pri_indev, s->sec_indev)) { + error_setg(errp, "'indev' and 'outdev' could not be same " + "for compare module"); + return; + } + + s->chr_pri_in = qemu_chr_find(s->pri_indev); + if (s->chr_pri_in == NULL) { + error_setg(errp, "Primary IN Device '%s' not found", + s->pri_indev); + return; + } + + /* inspect chardev opts */ + memset(&props, 0, sizeof(props)); + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) { + return; + } + + if (!props.is_socket || !props.is_unix) { + error_setg(errp, "chardev \"%s\" is not a unix socket", + s->pri_indev); + return; + } + + s->chr_sec_in = qemu_chr_find(s->sec_indev); + if (s->chr_sec_in == NULL) { + error_setg(errp, "Secondary IN Device '%s' not found", + s->sec_indev); + return; + } + + memset(&props, 0, sizeof(props)); + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) { + return; + } + + if (!props.is_socket || !props.is_unix) { + error_setg(errp, "chardev \"%s\" is not a unix socket", + s->sec_indev); + return; + } + + s->chr_out = qemu_chr_find(s->outdev); + if (s->chr_out == NULL) { + error_setg(errp, "OUT Device '%s' not found", s->outdev); + return; + } + + memset(&props, 0, sizeof(props)); + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) { + return; + } + + if (!props.is_socket || !props.is_unix) { + error_setg(errp, "chardev \"%s\" is not a unix socket", + s->outdev); + return; + } + + qemu_chr_fe_claim_no_fail(s->chr_pri_in); + + qemu_chr_fe_claim_no_fail(s->chr_sec_in); + + qemu_chr_fe_claim_no_fail(s->chr_out); + + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); + + return; +} + +static void colo_compare_class_init(ObjectClass *oc, void *data) +{ + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); + + ucc->complete = colo_compare_complete; +} + +static void colo_compare_init(Object *obj) +{ + object_property_add_str(obj, "primary_in", + compare_get_pri_indev, compare_set_pri_indev, + NULL); + object_property_add_str(obj, "secondary_in", + compare_get_sec_indev, compare_set_sec_indev, + NULL); + object_property_add_str(obj, "outdev", + compare_get_outdev, compare_set_outdev, + NULL); +} + +static void colo_compare_finalize(Object *obj) +{ + CompareState *s = COLO_COMPARE(obj); + + if (s->chr_pri_in) { + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); + qemu_chr_fe_release(s->chr_pri_in); + } + if (s->chr_sec_in) { + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); + qemu_chr_fe_release(s->chr_sec_in); + } + if (s->chr_out) { + qemu_chr_fe_release(s->chr_out); + } + + g_free(s->pri_indev); + g_free(s->sec_indev); + g_free(s->outdev); +} + +static const TypeInfo colo_compare_info = { + .name = TYPE_COLO_COMPARE, + .parent = TYPE_OBJECT, + .instance_size = sizeof(CompareState), + .instance_init = colo_compare_init, + .instance_finalize = colo_compare_finalize, + .class_size = sizeof(CompareClass), + .class_init = colo_compare_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_USER_CREATABLE }, + { } + } +}; + +static void register_types(void) +{ + type_register_static(&colo_compare_info); +} + +type_init(register_types); diff --git a/qemu-options.hx b/qemu-options.hx index 587de8f..33d5d0b 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by The file format is libpcap, so it can be analyzed with tools such as tcpdump or Wireshark. +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, +outdev=@var{chardevid} + +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with +secondary packet. If the packet same, we will output primary +packet to outdev@var{chardevid}, else we will notify colo-frame +do checkpoint and send primary packet to outdev@var{chardevid}. + +we can use it with the help of filter-mirror and filter-redirector. + +@example + +primary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 + +secondary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=red0,host=3.3.3.3,port=9003 +-chardev socket,id=red1,host=3.3.3.3,port=9004 +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + +@end example + +If you want to know the detail of above command line, you can read +the colo-compare git log. + @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] diff --git a/vl.c b/vl.c index cbe51ac..c6b9a6f 100644 --- a/vl.c +++ b/vl.c @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type) if (g_str_equal(type, "filter-buffer") || g_str_equal(type, "filter-dump") || g_str_equal(type, "filter-mirror") || - g_str_equal(type, "filter-redirector")) { + g_str_equal(type, "filter-redirector") || + g_str_equal(type, "colo-compare")) { return false; }
This a COLO net ascii figure: Primary qemu Secondary qemu +--------------------------------------------------------------+ +----------------------------------------------------------------+ | +----------------------------------------------------------+ | | +-----------------------------------------------------------+ | | | | | | | | | | | guest | | | | guest | | | | | | | | | | | +-------^--------------------------+-----------------------+ | | +---------------------+--------+----------------------------+ | | | | | | ^ | | | | | | | | | | | | +------------------------------------------------------+ | | | | |netfilter| | | | | | netfilter | | | | +----------+ +----------------------------+ | | | +-----------------------------------------------------------+ | | | | | | | out | | | | | | filter excute order | | | | | | +-----------------------------+ | | | | | | +-------------------> | | | | | | | | | | | | | | | | TCP | | | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| | | | +------------+ +---+----+---v+rewriter++ +------------+ | | | | | | | | | | |in | |in | | | | | | | | | | | | | | | | filter | | filter | | filter +------> colo <------+ +--------> filter +--> adjust | adjust +--> filter | | | | | | mirror | |redirector| |redirector| | | compare | | | | | | redirector | | ack | seq | | redirector | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +----^-----+ +----+-----+ +----------+ | +---------+ | | | | +------------+ +--------+--------------+ +---+--------+ | | | | | tx | rx rx | | | | | tx all | rx | | | | | | | | | | +-----------------------------------------------------------+ | | | | +--------------+ | | | | | | | | | filter excute order | | | | | | | | | | +----------------> | | | +--------------------------------------------------------+ | | +-----------------------------------------+ | | | | | | | | | +--------------------------------------------------------------+ +----------------------------------------------------------------+ |guest receive | guest send | | +--------+----------------------------v------------------------+ | | NOTE: filter direction is rx/tx/all | tap | rx:receive packets sent to the netdev | | tx:receive packets sent by the netdev +--------------------------------------------------------------+ In COLO-compare, we do packet comparing job. Packets coming from the primary char indev will be sent to outdev. Packets coming from the secondary char dev will be dropped after comparing. colo-comapre need two input chardev and one output chardev: primary_in=chardev1-id (source: primary send packet) secondary_in=chardev2-id (source: secondary send packet) outdev=chardev3-id usage: primary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 secondary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 -chardev socket,id=red0,host=3.3.3.3,port=9003 -chardev socket,id=red1,host=3.3.3.3,port=9004 -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com> --- net/Makefile.objs | 1 + net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++ qemu-options.hx | 39 ++++++++ vl.c | 3 +- 4 files changed, 326 insertions(+), 1 deletion(-) create mode 100644 net/colo-compare.c