From patchwork Mon Mar 25 22:20:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Young X-Patchwork-Id: 10870263 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E4AE925 for ; Mon, 25 Mar 2019 22:22:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41C9D291B4 for ; Mon, 25 Mar 2019 22:22:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3624929275; Mon, 25 Mar 2019 22:22:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 18D1E291B4 for ; Mon, 25 Mar 2019 22:22:44 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1h8Xwy-0000Kq-7c; Mon, 25 Mar 2019 22:20:12 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1h8Xww-0000Kl-NE for xen-devel@lists.xenproject.org; Mon, 25 Mar 2019 22:20:10 +0000 X-Inumbo-ID: 2584f91a-4f4c-11e9-bc90-bc764e045a96 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (unknown [2a01:111:f400:fe0d::704]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 2584f91a-4f4c-11e9-bc90-bc764e045a96; Mon, 25 Mar 2019 22:20:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=durhamuniversity.onmicrosoft.com; s=selector1-durham-ac-uk; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VnEPghwR4guZBHxrmRoZbxUDtCukip3f3L8WZa0IezQ=; b=VBiaDXzZPpBqosUxPNvsSJ11166Mc6cdBUQAvN8Jt9e/qTljmnI5v823yReA9zOOjRs0CuFyVGhNDt79SlC8l0EcwI4Sywfv75rfUTVeOqWK93S1HGm+V/aMiZMtM/W+2CzYYa0VEUTWJVdpFfI+hTlJ9iYWsrWPAbeoBFPlBII= Received: from VI1PR0602MB3582.eurprd06.prod.outlook.com (52.134.1.27) by VI1PR0602MB3789.eurprd06.prod.outlook.com (52.134.18.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.18; Mon, 25 Mar 2019 22:20:05 +0000 Received: from VI1PR0602MB3582.eurprd06.prod.outlook.com ([fe80::382f:c14a:cf37:3af4]) by VI1PR0602MB3582.eurprd06.prod.outlook.com ([fe80::382f:c14a:cf37:3af4%4]) with mapi id 15.20.1730.019; Mon, 25 Mar 2019 22:20:05 +0000 From: "YOUNG, MICHAEL A." To: xen-devel Thread-Topic: python3 issues Thread-Index: AQHU41jlI9hK+CwUjEaO+B/EmRSDgA== Date: Mon, 25 Mar 2019 22:20:05 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: user-agent: Alpine 2.21 (LFD 202 2017-01-01) x-originating-ip: [2a00:23c4:9208:400:c49a:db08:8036:b390] x-clientproxiedby: LO2P265CA0401.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:f::29) To VI1PR0602MB3582.eurprd06.prod.outlook.com (2603:10a6:803:2::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=m.a.young@durham.ac.uk; x-ms-exchange-messagesentrepresentingtype: 1 x-x-sender: michael@austen3.home x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5d33fb56-1fd1-4ea3-5236-08d6b1700846 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(49563074)(7193020); SRVR:VI1PR0602MB3789; x-ms-traffictypediagnostic: VI1PR0602MB3789: x-microsoft-antispam-prvs: x-forefront-prvs: 0987ACA2E2 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(346002)(39860400002)(366004)(376002)(136003)(199004)(189003)(786003)(6506007)(53936002)(316002)(81156014)(6916009)(6486002)(53346004)(68736007)(386003)(6436002)(478600001)(99936001)(46003)(6512007)(9686003)(5660300002)(52116002)(2906002)(102836004)(86362001)(14454004)(105586002)(106356001)(5024004)(97736004)(8936002)(7736002)(99286004)(486006)(256004)(305945005)(4744005)(58126008)(25786009)(476003)(71200400001)(71190400001)(186003)(7116003)(6116002)(8676002)(36756003)(81166006)(74482002); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0602MB3789; H:VI1PR0602MB3582.eurprd06.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: durham.ac.uk does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: j6sgARBAk95oEVrMfBvAvlLCHJMPfo6QwBdSIL/CVTISDb+qPMuU+Mwb2CqK9q5d9piBczLusSPxAiixo/Wk76a22Ug6VSdba2ZbBDJggL4AVvGZz1SmeFvc+j3AYtvBG5eWHMrd58cS/5USeMIYjBWGeXfH1GlE7qmk4nZ8UKTXDV0VrYVSS6fGHZlYqTW4WjWKzyvuYcyJgTaTwxOjgQxwSotQgM9R0wm0GBg6H5qCgwB2xesfOKhmw9VTvYie0OycFDLFwf1QJSyrePs1PV9LyB0Oq06ZMKA4sjLXkdFJlYZOhm7R6e62Lhb1ZcwzzA1pCCrUvqyVrz4UGV2UEQ3lXK104sKer3rJT1gtH8HeDZNRC7Vn4ilBHeNWAfOy937azD0778Yb+y9Ueu58jtr+TMA+C+69LaSGofyobv0= MIME-Version: 1.0 X-OriginatorOrg: durham.ac.uk X-MS-Exchange-CrossTenant-Network-Message-Id: 5d33fb56-1fd1-4ea3-5236-08d6b1700846 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Mar 2019 22:20:05.4805 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7250d88b-4b68-4529-be44-d59a2d8a6f94 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0602MB3789 Subject: [Xen-devel] python3 issues X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP I have been testing the python3 changes committed to xen and found a few issues. There are a couple of ocaml python build scripts that don't work for me with python3, and I needed a few fixes to get pygrub to work, mostly due to the change from strings to bytes. I am attaching the patch I put together in testing to get these things to work to illustrate where the problems are and in case it is useful to others, though I believe at least some of it isn't compatible with python2. Michael Young --- xen-4.12.0-rc5/tools/ocaml/libs/xentoollog/genlevels.py.orig 2019-03-06 14:42:49.000000000 +0000 +++ xen-4.12.0-rc5/tools/ocaml/libs/xentoollog/genlevels.py 2019-03-13 21:33:59.805930989 +0000 @@ -1,6 +1,7 @@ #!/usr/bin/python import sys +from functools import reduce def read_levels(): f = open('../../../libs/toollog/include/xentoollog.h', 'r') @@ -86,14 +87,14 @@ def autogen_header(open_comment, close_comment): s = open_comment + " AUTO-GENERATED FILE DO NOT EDIT " + close_comment + "\n" s += open_comment + " autogenerated by \n" - s += reduce(lambda x,y: x + " ", range(len(open_comment + " ")), "") + s += reduce(lambda x,y: x + " ", list(range(len(open_comment + " "))), "") s += "%s" % " ".join(sys.argv) s += "\n " + close_comment + "\n\n" return s if __name__ == '__main__': if len(sys.argv) < 3: - print >>sys.stderr, "Usage: genlevels.py " + print("Usage: genlevels.py ", file=sys.stderr) sys.exit(1) levels, olevels = read_levels() --- xen-4.12.0-rc5/tools/ocaml/libs/xl/genwrap.py.orig 2019-03-06 14:42:49.000000000 +0000 +++ xen-4.12.0-rc5/tools/ocaml/libs/xl/genwrap.py 2019-03-13 21:34:00.674962832 +0000 @@ -3,6 +3,7 @@ import sys,os import idl +from functools import reduce # typename -> ( ocaml_type, c_from_ocaml, ocaml_from_c ) builtins = { @@ -78,7 +79,7 @@ elif isinstance(ty,idl.Array): return "%s array" % ocaml_type_of(ty.elem_type) elif isinstance(ty,idl.Builtin): - if not builtins.has_key(ty.typename): + if ty.typename not in builtins: raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty))) typename,_,_ = builtins[ty.typename] if not typename: @@ -251,7 +252,7 @@ else: s += "\texternal default : ctx -> %sunit -> t = \"stub_libxl_%s_init\"\n" % (union_args, ty.rawname) - if functions.has_key(ty.rawname): + if ty.rawname in functions: for name,args in functions[ty.rawname]: s += "\texternal %s : " % name s += " -> ".join(args) @@ -278,7 +279,7 @@ else: s += "%s = Int_val(%s);" % (c, o) elif isinstance(ty,idl.Builtin): - if not builtins.has_key(ty.typename): + if ty.typename not in builtins: raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty))) _,fn,_ = builtins[ty.typename] if not fn: @@ -375,7 +376,7 @@ else: s += "%s = Val_int(%s);" % (o, c) elif isinstance(ty,idl.Builtin): - if not builtins.has_key(ty.typename): + if ty.typename not in builtins: raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty))) _,_,fn = builtins[ty.typename] if not fn: @@ -513,14 +514,14 @@ def autogen_header(open_comment, close_comment): s = open_comment + " AUTO-GENERATED FILE DO NOT EDIT " + close_comment + "\n" s += open_comment + " autogenerated by \n" - s += reduce(lambda x,y: x + " ", range(len(open_comment + " ")), "") + s += reduce(lambda x,y: x + " ", list(range(len(open_comment + " "))), "") s += "%s" % " ".join(sys.argv) s += "\n " + close_comment + "\n\n" return s if __name__ == '__main__': if len(sys.argv) < 4: - print >>sys.stderr, "Usage: genwrap.py " + print("Usage: genwrap.py ", file=sys.stderr) sys.exit(1) (_,types) = idl.parse(sys.argv[1]) @@ -533,7 +534,7 @@ for t in blacklist: if t not in [ty.rawname for ty in types]: - print "unknown type %s in blacklist" % t + print("unknown type %s in blacklist" % t) types = [ty for ty in types if not ty.rawname in blacklist] @@ -564,7 +565,7 @@ cinc.write("\n") cinc.write(gen_Val_ocaml(ty)) cinc.write("\n") - if functions.has_key(ty.rawname): + if ty.rawname in functions: cinc.write(gen_c_stub_prototype(ty, functions[ty.rawname])) cinc.write("\n") if ty.init_fn is not None: --- xen-4.12.0-rc6/tools/pygrub/src/GrubConf.py.orig 2019-03-24 22:44:05.502581989 +0000 +++ xen-4.12.0-rc6/tools/pygrub/src/GrubConf.py 2019-03-24 22:49:14.025934786 +0000 @@ -230,10 +230,10 @@ def _get_default(self): return self._default def _set_default(self, val): - if val == "saved": + if val == "saved" or not val.isdecimal(): self._default = 0 else: - self._default = val + self._default = int(val) if self._default < 0: raise ValueError("default must be positive number") --- xen-4.12.0-rc6/tools/pygrub/src/pygrub.orig 2019-03-24 22:44:05.503582025 +0000 +++ xen-4.12.0-rc6/tools/pygrub/src/pygrub 2019-03-24 22:48:24.446113809 +0000 @@ -457,7 +457,7 @@ # limit read size to avoid pathological cases buf = f.read(FS_READ_MAX) del f - self.cf.parse(buf) + self.cf.parse(buf.decode()) def image_index(self): if isinstance(self.cf.default, int): @@ -960,5 +960,5 @@ ostring = format_simple(bootcfg["kernel"], bootcfg["ramdisk"], args, "\0") sys.stdout.flush() - os.write(fd, ostring) + os.write(fd, ostring.encode())