From patchwork Sat Dec 7 00:33:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23CE7109A for ; Sat, 7 Dec 2019 00:33:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 002CA21835 for ; Sat, 7 Dec 2019 00:33:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="ruiNDs87" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726474AbfLGAdr (ORCPT ); Fri, 6 Dec 2019 19:33:47 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:36308 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726388AbfLGAdq (ORCPT ); Fri, 6 Dec 2019 19:33:46 -0500 Received: by mail-pl1-f193.google.com with SMTP id k20so3409528pls.3 for ; Fri, 06 Dec 2019 16:33:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vbE+YDAzXoZ3rSFqVfVwZCsgRh7R2WBbV1iIGX0bo+c=; b=ruiNDs87aEngud/Lo4tZ2ajUet3h0pUt617apr8tpqt/I4kkkxZzjs1SyFdj900jrU g5TfgFP4CWK0uElF4zXrQoYxXNXJqhzXxA1Ku1L2Dq5NmhjkljWK8owR6PGHA8nqPuNY 4LR2Nas0djAtSIbxQDxdYu0DLR74Ry7spNoqz+gRgkorhDOJpqetQNsf7n2Aq65OtCOw i2LOY2oM1Li2cjOZe7Nf3PNiiHHli//lFHWOBg6Um0TtRiurB/imQeJllwXpQhkQWwSc GCsAG3aYICSzw9jq0ThRjH7v/Llon4RFJqfrQOohifZWs3mUfeDf9bou9z43j9N2TF9Y 2lAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vbE+YDAzXoZ3rSFqVfVwZCsgRh7R2WBbV1iIGX0bo+c=; b=ZQHHwF/fNJW8dUsFLk0wQvzeTFAtv9yil944Cg458VBnKJKcRR+YUW0i9N1d0SlKEF 2e831YoZ+fWQIJP31sF4ieXEm8zzjN123QQOEMR70RmTahYkuo5mKAzCCCiFVdrvVSwg BIXQ4kQlA7D7J6v9C5ZrEJ+oJX40rGf32CyQcJBDfod8+HkUfPiNizykxNhqGZd3uP7r HsHTg4+5ThmIe+jx9Icc+8HdfkdACWxfEZq+609DeoY/OvnWLt6RG65oJGlrSwnMeK7C UJDxkHPiUb1yZuMEn0XYxRTvmgP/hBlGQXTff+y9QIacr8gxunMXzTGHAr3wIcVRNN3K 9w1A== X-Gm-Message-State: APjAAAVw1fS2J9GEpp4D/5fXPphWCDeSt7KLS7cky+WHwCeYQ0mL6MG1 mhD4V6K3G9izktAjU0P+CV1rrrsfOpFGrw== X-Google-Smtp-Source: APXvYqx89y9MBhacuTs4uzj9eQkc4oCukeiiy8CrKuFmvkRyoCBZsahz4CLCfiI9jtyHzOSInH1d0Q== X-Received: by 2002:a17:90a:8a98:: with SMTP id x24mr19408084pjn.113.1575678825429; Fri, 06 Dec 2019 16:33:45 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:44 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 01/13] ci: also run linux-gcc pipeline with python-3.7 environment Date: Fri, 6 Dec 2019 16:33:19 -0800 Message-Id: <20191207003333.3228-2-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git-p4.py includes support for python-3, but this was not previously validated in CI. Lets actually do that. There is no tangible benefit to repeating python-3 tests for all environments, so only limit it to linux-gcc for now. Signed-off-by: Yang Zhao --- I assert that we don't need to run python3 tests on more platforms, but is this actually reasonable? azure-pipelines.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 37ed7e06c6..d5f9413248 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -331,7 +331,18 @@ jobs: displayName: linux-gcc condition: succeeded() pool: Hosted Ubuntu 1604 + strategy: + matrix: + python27: + python.version: '2.7' + python37: + python.version: '3.7' steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: '$(python.version)' + - bash: | + echo "##vso[task.setvariable variable=python_path]$(which python)" - bash: | test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 From patchwork Sat Dec 7 00:33:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9D4A109A for ; Sat, 7 Dec 2019 00:33:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C706F21835 for ; Sat, 7 Dec 2019 00:33:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="j2O1raRY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726491AbfLGAds (ORCPT ); Fri, 6 Dec 2019 19:33:48 -0500 Received: from mail-pj1-f54.google.com ([209.85.216.54]:45572 "EHLO mail-pj1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726388AbfLGAdr (ORCPT ); Fri, 6 Dec 2019 19:33:47 -0500 Received: by mail-pj1-f54.google.com with SMTP id r11so3424976pjp.12 for ; Fri, 06 Dec 2019 16:33:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8CADdXLVahndlaqvmeSNPYEQFyOBkF+EvJf8IL5MDjU=; b=j2O1raRYyKe2aXy8P8O9b8iKuOqYmtMKnOb/NBesXBbV0FsFfbJ2Bz10BEJ9tGR7yP a1G4mCOWwOgPy/ZWfrX5J3Kypr4EhiFgrY1Y3Rk39yTf05h3Q0Q1EVvLkKZvXvaD/1op LEakac35cwaqdOep/qqOVOWpUZwgs/OmpGE2Qp7G4qBdh+FB16CeOX48Fc8QVNeWwqoX ya3gfI/prUAcfpOQVwDLgB8IVCz9Ek5f5/7VQY2wRz4U4iRywg32iwzJBuev55RS5u3t RGJ9TkPFQK0c11OvcM4/CXIUnscAfjjx8KBzALPGBTbXdP16JEVF8TgDGJCDuGDl1Cnk 97qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8CADdXLVahndlaqvmeSNPYEQFyOBkF+EvJf8IL5MDjU=; b=gz+d62JUz5/Mqv0oLQdWhFC2TEnshcfQdwcIFkKYXihwDjvnIOhzEa1HpZ/1uUjSNX HOBpj0Ui8AuqmsT8l6030NWHoK842F4E3bOCvgKdSnzPYyqAVGCGQ7cBL1V6INYiEJUe Y7U0iTyOg2PCQZslJmtmN51kIZUuN+OKKfQLpRkl+S3ZUjYjUijkoOeyGRLUO3i7bed2 sryjiHQgAJFvn/IdOaEnGifgtmrUIjlpCImLPwp/PtFg/qdNCIion4npLGW6AlGI4xEo Fv/gKRc3coAusFdgeaiqPwp/aV7ecUVCC8JpOvpW6WOa+u/vt5fwQlJUJQo5x9np51E7 hr2w== X-Gm-Message-State: APjAAAVqsaDh+AUai4XOy1a4tPklCMHgCt/zF3D9Rys9QIh7iyyCR9m4 pOKCLnXaFNyuYgLtRN8rB36JB84jN52Zsw== X-Google-Smtp-Source: APXvYqx3i1u7mbRvYSbjt6BVGjIfO3+NVeLTbdkRs0L1m2tjM+v3yOqjg+XfIAc/q3Mm+rZDrQaaTQ== X-Received: by 2002:a17:90a:974a:: with SMTP id i10mr19415780pjw.0.1575678826642; Fri, 06 Dec 2019 16:33:46 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:45 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 02/13] git-p4: make python-2.7 the oldest supported version Date: Fri, 6 Dec 2019 16:33:20 -0800 Message-Id: <20191207003333.3228-3-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Python-2.6 and earlier have been end-of-life'd for many years now, and we actually already use 2.7-only features in the code. Make the version check reflect current realities. --- git-p4.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/git-p4.py b/git-p4.py index 60c73b6a37..d8f88884db 100755 --- a/git-p4.py +++ b/git-p4.py @@ -8,9 +8,8 @@ # License: MIT # import sys -if sys.hexversion < 0x02040000: - # The limiter is the subprocess module - sys.stderr.write("git-p4: requires Python 2.4 or later.\n") +if sys.version_info.major < 3 and sys.version_info.minor < 7: + sys.stderr.write("git-p4: requires Python 2.7 or later.\n") sys.exit(1) import os import optparse From patchwork Sat Dec 7 00:33:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 702426C1 for ; Sat, 7 Dec 2019 00:33:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4CA862245C for ; Sat, 7 Dec 2019 00:33:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="uB4/UPuQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726552AbfLGAdu (ORCPT ); Fri, 6 Dec 2019 19:33:50 -0500 Received: from mail-pl1-f182.google.com ([209.85.214.182]:37207 "EHLO mail-pl1-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726479AbfLGAds (ORCPT ); Fri, 6 Dec 2019 19:33:48 -0500 Received: by mail-pl1-f182.google.com with SMTP id bb5so3407342plb.4 for ; Fri, 06 Dec 2019 16:33:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J1FxdPV0xnSaQlKxxVPFCu4Q068GmvCoutzbjsb8U5M=; b=uB4/UPuQup5AKM28YecsZhUJ68n7/QAfGEf2ZCAxt5TSZ53L2TJGzdohI//13gsKND aoZl0GspXQFEwTnXcVFsrlxkmmx76yJo5vbtB4B24ejo6JZITE/8f65Ztx07yUKkt5k2 MUuHg1aIfoTVe2mkWEsPNcj9/71DqnKXCGzbclOaDUn/kxf0KTIk0unozA/wovStkw0t ZnmLmZ+5KYLghIKCtrseMauaAXy05iFNwgseLFv8w952FbM/d3TcpQYkVOySsHESxXie CDMuN8QfhT4nMkZlRLvgJsLlDGV7v73a5qcYj9vm10y+p118YlTW9jTj3qNDKq2MZ3RU Ojtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J1FxdPV0xnSaQlKxxVPFCu4Q068GmvCoutzbjsb8U5M=; b=dJvLzjJWTTA9ORX1JZ1hxxXOiSxwYmAOtXCkW6gv6bu1puJ1nRWrW5UxpAR9YFToDh xwUkkLTeLbVGR6jBnfZnBJjTtB3KMXeBrCQUh2Ss0h4lFj6buD1J0hP/cGHJAH3m7zy8 Tg1ZzU1TjJZRdvDsyk5HfcrgfNZ9j4sjwRDF67Vj3EVuzDkgnjPs+kt+2vRcYlajiY57 lgmahtnfvxCRJDuHr5F6+4LymqHbxv+zHZaRZv6Ou+EaqyrtBi40ctSD8gExMDHZSoZS ly2W6Tqv2dW3p6IPN8JIJMzpz8FkDYMD/rxYP8xoqJDEA6wPVicJsTqYbMRLCXmBDA2c 5LbA== X-Gm-Message-State: APjAAAUpjtUNg9n1U6HFDmziH5E8jAgI17qV5/itn4QLVNlyBvTceCV4 mn3bvHWva5fP5HnKaHz2mw5UAaoeOly0TA== X-Google-Smtp-Source: APXvYqxLEG2chrLZY0cpdQZf2vZCSOxi9E25YraOJqqKHZprT9lbr65STuGM3AAdGhGPQxB08O4eEQ== X-Received: by 2002:a17:90a:1b6b:: with SMTP id q98mr19091495pjq.106.1575678827564; Fri, 06 Dec 2019 16:33:47 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:47 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 03/13] git-p4: simplify python version detection Date: Fri, 6 Dec 2019 16:33:21 -0800 Message-Id: <20191207003333.3228-4-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Instead of type shenanigans, just check the version object. Signed-off-by: Yang Zhao --- git-p4.py | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/git-p4.py b/git-p4.py index d8f88884db..ebeef35a92 100755 --- a/git-p4.py +++ b/git-p4.py @@ -27,18 +27,9 @@ import errno # support basestring in python3 -try: - unicode = unicode -except NameError: - # 'unicode' is undefined, must be Python 3 - str = str - unicode = str - bytes = bytes +if sys.version_info.major >= 3: basestring = (str,bytes) else: - # 'unicode' exists, must be Python 2 - str = str - unicode = unicode bytes = str basestring = basestring From patchwork Sat Dec 7 00:33:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F02F4109A for ; Sat, 7 Dec 2019 00:33:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B6B6D205ED for ; Sat, 7 Dec 2019 00:33:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="bLNM1ocq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726595AbfLGAdx (ORCPT ); Fri, 6 Dec 2019 19:33:53 -0500 Received: from mail-pl1-f182.google.com ([209.85.214.182]:34518 "EHLO mail-pl1-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726388AbfLGAdt (ORCPT ); Fri, 6 Dec 2019 19:33:49 -0500 Received: by mail-pl1-f182.google.com with SMTP id h13so3419908plr.1 for ; Fri, 06 Dec 2019 16:33:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9FBp/3+bkK9ifG9OGAMf1ZSZQg355B0N36vipH8avig=; b=bLNM1ocq3euJyzBj/HtLtpNdQrJIzLL+U95PEXpHX7xJyaFVA0V240qprJS/mVuYxz DcGzVjYv9zUBGeZiiZ80fiOdE6V3G4yq00MR/Y0Xhy1qmAuw8EvIYvJOcLfIQs4wI6hd fFNg//zcthNIv/+a2RlVPOgxVwl8JZF2RyOucc1kbAO/zCQnVIQyesKnrTTyrJCFpjTh ybqcmPU6ykyzIxgBLW9brBfds1bTgj6z2mRrDfjJAuXf5hyEsRw1DuYOEadj5n4UwNm0 qTg9xo76OpBPHtbiV7Bqn9L4X8m+kXzc80+HQZGn6uTni/CZwaFx0JB6Z3TiJlVDNmz7 6MJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9FBp/3+bkK9ifG9OGAMf1ZSZQg355B0N36vipH8avig=; b=j3/pr1a1kuIDsdjRdNPMCD/i0HOS4M/+0MXyYjw/rR9sVeW3/FWvJZbKz+xF3dKo9K g0eUzsYca8eOlEp1za+PgSUXKQAqCNn7itC87LatCG2nuyD0m5BxzjAVJAtKwULUDd2/ 206WSW0MRPPvicyv8Jz+vw5isBnzar0hQ8LddYv+byHDtOX2lnZPVwHPKN/X/45KLlaR K4sYag4qogekAG81HkOTn2lVrNz9uPCySd7ixJvaPd2F7o2dpeHeLkcb4Uz5QcX91IIk zcyNv1siwLPEarEOVLryCrOKFqAMko90CDeRZiGonkPIsh3BgwkkEBMAcNxGF8AWyVaO 3WRA== X-Gm-Message-State: APjAAAXiFppSpR4aoD1PVJ7txToXmkz+Q9/LC37rZ1Zu8FPmUzro7Zw7 ssu0aYVpvey4gAyN9sbyGHMuzg9Ij1mCeA== X-Google-Smtp-Source: APXvYqxB9moN0JJME9aXMYUvaZA9wzYCcMa1OGoGl/q0m4v620bnpIVW3IEpxZU/A+qIv8kbf3HDXg== X-Received: by 2002:a17:90a:d985:: with SMTP id d5mr17287985pjv.73.1575678828527; Fri, 06 Dec 2019 16:33:48 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:47 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 04/13] git-p4: decode response from p4 to str for python3 Date: Fri, 6 Dec 2019 16:33:22 -0800 Message-Id: <20191207003333.3228-5-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The marshalled dict in the response given on STDOUT by p4 uses `str` for keys and string values. When run using python3, these values are deserialized as `bytes`, leading to a whole host of problems as the rest of the code assumes `str` is used throughout. This patch changes the deserialization behaviour such that, as much as possible, text output from p4 is decoded to native unicode strings. Exceptions are made for the field `data` as it is usually arbitrary binary data. `depotFile[0-9]*`, `path`, and `clientFile` are also exempt as they contain path information which may not be UTF-8 encoding compatible, and must survive round-trip back to p4. Signed-off-by: Yang Zhao SQUASH: use unicode string internally throughout --- git-p4.py | 61 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 45 insertions(+), 16 deletions(-) diff --git a/git-p4.py b/git-p4.py index ebeef35a92..6720c7b24a 100755 --- a/git-p4.py +++ b/git-p4.py @@ -157,6 +157,19 @@ def die(msg): sys.stderr.write(msg + "\n") sys.exit(1) +# We need different encoding/decoding strategies for text data being passed +# around in pipes depending on python version +if sys.version_info.major >= 3: + def decode_text_stream(s): + return s.decode() if isinstance(s, bytes) else s + def encode_text_stream(s): + return s.encode() if isinstance(s, str) else s +else: + def decode_text_stream(s): + return s + def encode_text_stream(s): + return s.encode('utf_8') if isinstance(s, unicode) else s + def write_pipe(c, stdin): if verbose: sys.stderr.write('Writing pipe: %s\n' % str(c)) @@ -186,7 +199,7 @@ def read_pipe_full(c): expand = isinstance(c,basestring) p = subprocess.Popen(c, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=expand) (out, err) = p.communicate() - return (p.returncode, out, err) + return (p.returncode, out, decode_text_stream(err)) def read_pipe(c, ignore_error=False): """ Read output from command. Returns the output text on @@ -209,11 +222,11 @@ def read_pipe_text(c): if retcode != 0: return None else: - return out.rstrip() + return decode_text_stream(out).rstrip() -def p4_read_pipe(c, ignore_error=False): +def p4_read_pipe(c, ignore_error=False, raw=False): real_cmd = p4_build_cmd(c) - return read_pipe(real_cmd, ignore_error) + return read_pipe(real_cmd, ignore_error, raw=raw) def read_pipe_lines(c): if verbose: @@ -222,7 +235,7 @@ def read_pipe_lines(c): expand = isinstance(c, basestring) p = subprocess.Popen(c, stdout=subprocess.PIPE, shell=expand) pipe = p.stdout - val = pipe.readlines() + val = [decode_text_stream(line) for line in pipe.readlines()] if pipe.close() or p.wait(): die('Command failed: %s' % str(c)) @@ -253,6 +266,7 @@ def p4_has_move_command(): cmd = p4_build_cmd(["move", "-k", "@from", "@to"]) p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) (out, err) = p.communicate() + err = decode_text_stream(err) # return code will be 1 in either case if err.find("Invalid option") >= 0: return False @@ -633,6 +647,20 @@ def p4CmdList(cmd, stdin=None, stdin_mode='w+b', cb=None, skip_info=False, try: while True: entry = marshal.load(p4.stdout) + if bytes is not str: + # Decode unmarshalled dict to use str keys and values, except for: + # - `data` which may contain arbitrary binary data + # - `depotFile[0-9]*`, `path`, or `clientFile` which may contain non-UTF8 encoded text + decoded_entry = {} + for key, value in entry.items(): + key = key.decode() + if isinstance(value, bytes) and not (key in ('data', 'path', 'clientFile') or key.startswith('depotFile')): + value = value.decode() + decoded_entry[key] = value + # Parse out data if it's an error response + if decoded_entry.get('code') == 'error' and 'data' in decoded_entry: + decoded_entry['data'] = decoded_entry['data'].decode() + entry = decoded_entry if skip_info: if 'code' in entry and entry['code'] == 'info': continue @@ -850,6 +878,7 @@ def branch_exists(branch): cmd = [ "git", "rev-parse", "--symbolic", "--verify", branch ] p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, _ = p.communicate() + out = decode_text_stream(out) if p.returncode: return False # expect exactly one line of output: the branch name @@ -1993,7 +2022,7 @@ def applyCommit(self, id): tmpFile = os.fdopen(handle, "w+b") if self.isWindows: submitTemplate = submitTemplate.replace("\n", "\r\n") - tmpFile.write(submitTemplate) + tmpFile.write(encode_text_stream(submitTemplate)) tmpFile.close() if self.prepare_p4_only: @@ -2040,11 +2069,11 @@ def applyCommit(self, id): if self.edit_template(fileName): # read the edited message and submit tmpFile = open(fileName, "rb") - message = tmpFile.read() + message = decode_text_stream(tmpFile.read()) tmpFile.close() if self.isWindows: message = message.replace("\r\n", "\n") - submitTemplate = message[:message.index(separatorLine)] + submitTemplate = encode_text_stream(message[:message.index(separatorLine)]) if update_shelve: p4_write_pipe(['shelve', '-r', '-i'], submitTemplate) @@ -2145,7 +2174,7 @@ def exportGitTags(self, gitTags): print("Not creating p4 label %s for tag due to option" \ " --prepare-p4-only" % name) else: - p4_write_pipe(["label", "-i"], labelTemplate) + p4_write_pipe(["label", "-i"], encode_text_stream(labelTemplate)) # Use the label p4_system(["tag", "-l", name] + @@ -2469,7 +2498,7 @@ def append(self, view_line): def convert_client_path(self, clientFile): # chop off //client/ part to make it relative - if not clientFile.startswith(self.client_prefix): + if not decode_path(clientFile).startswith(self.client_prefix): die("No prefix '%s' on clientFile '%s'" % (self.client_prefix, clientFile)) return clientFile[len(self.client_prefix):] @@ -2729,7 +2758,7 @@ def splitFilesIntoBranches(self, commit): return branches def writeToGitStream(self, gitMode, relPath, contents): - self.gitStream.write('M %s inline %s\n' % (gitMode, relPath)) + self.gitStream.write(encode_text_stream(u'M {} inline {}\n'.format(gitMode, relPath))) self.gitStream.write('data %d\n' % sum(len(d) for d in contents)) for d in contents: self.gitStream.write(d) @@ -2770,7 +2799,7 @@ def streamOneP4File(self, file, contents): git_mode = "120000" # p4 print on a symlink sometimes contains "target\n"; # if it does, remove the newline - data = ''.join(contents) + data = ''.join(decode_text_stream(c) for c in contents) if not data: # Some version of p4 allowed creating a symlink that pointed # to nothing. This causes p4 errors when checking out such @@ -2824,7 +2853,7 @@ def streamOneP4File(self, file, contents): pattern = p4_keywords_regexp_for_type(type_base, type_mods) if pattern: regexp = re.compile(pattern, re.VERBOSE) - text = ''.join(contents) + text = ''.join(decode_text_stream(c) for c in contents) text = regexp.sub(r'$\1$', text) contents = [ text ] @@ -2839,7 +2868,7 @@ def streamOneP4Deletion(self, file): if verbose: sys.stdout.write("delete %s\n" % relPath) sys.stdout.flush() - self.gitStream.write("D %s\n" % relPath) + self.gitStream.write(encode_text_stream(u'D {}\n'.format(relPath))) if self.largeFileSystem and self.largeFileSystem.isLargeFile(relPath): self.largeFileSystem.removeLargeFile(relPath) @@ -2939,9 +2968,9 @@ def streamP4FilesCbSelf(entry): if 'shelved_cl' in f: # Handle shelved CLs using the "p4 print file@=N" syntax to print # the contents - fileArg = '%s@=%d' % (f['path'], f['shelved_cl']) + fileArg = f['path'] + encode_text_stream('@={}'.format(f['shelved_cl'])) else: - fileArg = '%s#%s' % (f['path'], f['rev']) + fileArg = f['path'] + encode_text_stream('#{}'.format(f['rev'])) fileArgs.append(fileArg) From patchwork Sat Dec 7 00:33:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277221 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8357E14B7 for ; Sat, 7 Dec 2019 00:33:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6027D2245C for ; Sat, 7 Dec 2019 00:33:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="W3Wu8vzj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726584AbfLGAdx (ORCPT ); Fri, 6 Dec 2019 19:33:53 -0500 Received: from mail-pf1-f182.google.com ([209.85.210.182]:41041 "EHLO mail-pf1-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726551AbfLGAdu (ORCPT ); Fri, 6 Dec 2019 19:33:50 -0500 Received: by mail-pf1-f182.google.com with SMTP id s18so4184621pfd.8 for ; Fri, 06 Dec 2019 16:33:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KhQ1YTQyFSHt1kjeeSmimqG1rGO2PgRR7vaP9hAVFGI=; b=W3Wu8vzjsQwodwx/+K+/a9q43hN+HRJElZJbR1rO6bGs4WXB0/fDewBlOoyxhqdNsb gKHJkMPTXolLZUrw8JpcQYiMD1BTxj/yZ3fBLqMgZlp1aFBUx2eUB1FZlTIXs3Tfhcuj 5iA8HPIS2UckKSi2Rx7Yjep52FKXIbKQ0t3nsOYeurgAIXuphgNADJc7t+7QMwpNMvMJ RWNxMMA3cWugjozAs0UXcnv+KKpWJYEnOrcS/EVmQJkaxL51niPxdaHmVdgicV6C5Tzy H3GiASNjTyxa7lK4N5xbtRDLC0qX2JNU8+WtEDQ2QSafT6aMewQEAHn9CwjXJ+fpz/9O weLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KhQ1YTQyFSHt1kjeeSmimqG1rGO2PgRR7vaP9hAVFGI=; b=YXQI/LCkpS32ptSLulhdx6W3huWeqqlPtoAiQeFpPD7jgtLivhMk0eXk+R0sYIXkiR lMq94hnyQCj+YbAu3rv6eJxfVN2ImSS71Mk/oCnRbgU16XMrv5XlEI5xi4BQI2yBeOAc vEOcey+jTYxt8GD5MVjw6NRuh8vdqfisoqOsTd2Xmw8rc9/QikQafXXrGbCwWeStwPxn VQpNoUn2QYLzxWa4SlZyvHiI4mHTYCuokei4gSULpiQi4IS+oG12mNbU0whjrZ2RahIC VzRgEzhfMxUOqXV/CG961+W89EwGpL1E7syk6sozGOj3N3hpOsf+ODmw4lKl1Vjj9a6W uAoA== X-Gm-Message-State: APjAAAXvuEaNdmPTIj26BkOEj8sgd+Fjvnd/dVcu7BvaQCrDjGZk68EQ iA/8YX2FP5Km9jN6Msw59q404rGZhMEd5w== X-Google-Smtp-Source: APXvYqw+yhYbg+6qXkZsYwL3hBMGrlXTN9WnxWtcMavMZg4pz3Z4zVzl2CKRPtNtTEH9RwHdfF0+7A== X-Received: by 2002:a63:b64a:: with SMTP id v10mr6694048pgt.145.1575678829481; Fri, 06 Dec 2019 16:33:49 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:48 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 05/13] git-p4: properly encode/decode communication with git for python 3 Date: Fri, 6 Dec 2019 16:33:23 -0800 Message-Id: <20191207003333.3228-6-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Under python3, calls to write() on the stream to `git fast-import` must be encoded. This patch wraps the IO object such that this encoding is done transparently. Conversely, any text data read from subprocesses must also be decoded before running through the rest of the pipeline. Signed-off-by: Yang Zhao --- git-p4.py | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/git-p4.py b/git-p4.py index 6720c7b24a..fefa716b17 100755 --- a/git-p4.py +++ b/git-p4.py @@ -201,10 +201,12 @@ def read_pipe_full(c): (out, err) = p.communicate() return (p.returncode, out, decode_text_stream(err)) -def read_pipe(c, ignore_error=False): +def read_pipe(c, ignore_error=False, raw=False): """ Read output from command. Returns the output text on success. On failure, terminates execution, unless ignore_error is True, when it returns an empty string. + + If raw is True, do not attempt to decode output text. """ (retcode, out, err) = read_pipe_full(c) if retcode != 0: @@ -212,6 +214,8 @@ def read_pipe(c, ignore_error=False): out = "" else: die('Command failed: %s\nError: %s' % (str(c), err)) + if not raw: + out = decode_text_stream(out) return out def read_pipe_text(c): @@ -238,7 +242,6 @@ def read_pipe_lines(c): val = [decode_text_stream(line) for line in pipe.readlines()] if pipe.close() or p.wait(): die('Command failed: %s' % str(c)) - return val def p4_read_pipe_lines(c): @@ -634,7 +637,8 @@ def p4CmdList(cmd, stdin=None, stdin_mode='w+b', cb=None, skip_info=False, stdin_file.write(stdin) else: for i in stdin: - stdin_file.write(i + '\n') + stdin_file.write(encode_text_stream(i)) + stdin_file.write(b'\n') stdin_file.flush() stdin_file.seek(0) @@ -3556,6 +3560,15 @@ def openStreams(self): self.gitStream = self.importProcess.stdin self.gitError = self.importProcess.stderr + if bytes is not str: + # Wrap gitStream.write() so that it can be called using `str` arguments + def make_encoded_write(write): + def encoded_write(s): + return write(s.encode() if isinstance(s, str) else s) + return encoded_write + + self.gitStream.write = make_encoded_write(self.gitStream.write) + def closeStreams(self): self.gitStream.close() if self.importProcess.wait() != 0: From patchwork Sat Dec 7 00:33:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277223 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4DE526C1 for ; Sat, 7 Dec 2019 00:33:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2B88821835 for ; Sat, 7 Dec 2019 00:33:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="XeDZF6nG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726599AbfLGAdy (ORCPT ); Fri, 6 Dec 2019 19:33:54 -0500 Received: from mail-pg1-f177.google.com ([209.85.215.177]:45754 "EHLO mail-pg1-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726479AbfLGAdx (ORCPT ); Fri, 6 Dec 2019 19:33:53 -0500 Received: by mail-pg1-f177.google.com with SMTP id b9so3738403pgk.12 for ; Fri, 06 Dec 2019 16:33:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8/26BeqPQxq5NRAsqzlKRP27DLVO1x3i1q0Qn0Vu3Io=; b=XeDZF6nGpyYMKYflDFW7VdCda/LkrOFIs2yN8eVdoQiq9T33CUu/YTFi6BQUV2e+pi qC6Ltf3OXR+FFJtPkSP0YtWM9wj7SWbGliVi5mKxqsCIP7w/3k5lULOEmI38kmykmYj1 jC0HIhYZaxNiMw73KmccEtrHOV9LnMpblyuHP6+5pCkR4tb01x27XqmAPereU0ym/WN5 N54M35pCpSVNhTQOYgZ7drO7zr8/0ST0ZQOFMz3lIXBEy//XYJiNwqKLEOlNCIKb2FJH vOWKbg/3fmXP4gP5SyyFi7LqsFpXdJVuwF+s+bhIhgK9fYNqi1HgDjR2O3tj3ubdv+s0 OZqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8/26BeqPQxq5NRAsqzlKRP27DLVO1x3i1q0Qn0Vu3Io=; b=cMYFpZR2duA9kcFAYM+Hj/V3pXuTG6NX67CrlO7uKf+VcdFIraNnTEmJwZC0+QiXSK rryVS/0qlibdWBTvKBC9CIChQ6raY+sFa0RU8TCyaSKzr+9OggBq/2yvJLz5B7Yd9J36 YD7NLQnxmdLaaER11li2R2LEua2mohvxFdXllN3gPGLm67VvyS95zJ76xn33TdZMINCw UPNTUTEolsHO8R4j8bxsqxSTRFktP4dHhNBkVTRC9A2xTLxXak0p4gHSUMAHz3O6Xy1V FKweO3U/UdpRjuCArqp8vm4f0SER24NiSLXm67W9PR1qo9gBUtQUhIVEmpkO8guwvglI JTAQ== X-Gm-Message-State: APjAAAVHHqFpj135HIwJEYGavzDmRTqLXGX3/g4fwRXMM0QT22nw1XS0 LUSZ/YsyQYjz49P9cHYSlds6LUV/082d3g== X-Google-Smtp-Source: APXvYqySL9fz/semqynXkWoYUnbTQrdSJ7Emi0Lul0/SQHpo0rb8M2jMVPST4xXNCLY/8IfZ0TTfMw== X-Received: by 2002:a63:ed4d:: with SMTP id m13mr6900125pgk.442.1575678831923; Fri, 06 Dec 2019 16:33:51 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:50 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 06/13] git-p4: open .gitp4-usercache.txt in text mode Date: Fri, 6 Dec 2019 16:33:25 -0800 Message-Id: <20191207003333.3228-8-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Opening .gitp4-usercache.txt in text mode makes python 3 happy without explicitly adding encoding and decoding. Signed-off-by: Yang Zhao --- git-p4.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/git-p4.py b/git-p4.py index c7d543b18e..bd3118e0e8 100755 --- a/git-p4.py +++ b/git-p4.py @@ -1402,14 +1402,14 @@ def getUserMapFromPerforceServer(self): for (key, val) in self.users.items(): s += "%s\t%s\n" % (key.expandtabs(1), val.expandtabs(1)) - open(self.getUserCacheFilename(), "wb").write(s) + open(self.getUserCacheFilename(), 'w').write(s) self.userMapFromPerforceServer = True def loadUserMapFromCache(self): self.users = {} self.userMapFromPerforceServer = False try: - cache = open(self.getUserCacheFilename(), "rb") + cache = open(self.getUserCacheFilename(), 'r') lines = cache.readlines() cache.close() for line in lines: From patchwork Sat Dec 7 00:33:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F680109A for ; Sat, 7 Dec 2019 00:33:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6087C21835 for ; Sat, 7 Dec 2019 00:33:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="bNzFdRJe" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726640AbfLGAd5 (ORCPT ); Fri, 6 Dec 2019 19:33:57 -0500 Received: from mail-pj1-f41.google.com ([209.85.216.41]:46378 "EHLO mail-pj1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726596AbfLGAdy (ORCPT ); Fri, 6 Dec 2019 19:33:54 -0500 Received: by mail-pj1-f41.google.com with SMTP id z21so3418323pjq.13 for ; Fri, 06 Dec 2019 16:33:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=quCq/FsTvEw32e8liA4eVaBcZqNsNHxOUevrfUaJBLI=; b=bNzFdRJegEHJsDPVLnpT/G4ChV41ZlKk0rTZNB5rnfcW3tQy1/P3Dako1qJTxSdxK2 YEt5f64UVZTaGIBpZm+nidbC+1k63Qqty8LzimxuZZos2ntO61DJKcTQZ/RDnuEzJQ/N bvIXLEccmznM16y64v2rr4wu3LE0W2Ud8AGMV/cm6u37UHZchMu5/RQTfL3Pl3l4mCu+ sOxz3J5a1QDO+FSgPQyV9qIObCTDS1kuOh5IuGsakJgBydNJoHHF8Wb8pNijSui2SsNp HDkZW1de4OMuTRdPfKCLN7Bv07vgzleO5yMAxFAiWnOhEzx06PHRHb2idUMuZtFARsJz nIgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=quCq/FsTvEw32e8liA4eVaBcZqNsNHxOUevrfUaJBLI=; b=fG6hntLcIRY9xLTqYkaG1l/dH5HKa9N0qrdSzsSp+EtAsYgmjElmPXeEw/by4glq1B JtWeLre86IsHSkXBUIYoMCs7eutHoDNF63Blquc9dZqAL/kVi3D2uC+udSHP4XYZPyti AdBOb+8FxVSEsQ4ppcoyiIZMICj/lIReeN80jcFXI9p5QWHxgygatKxvf7h4THWyZuGD BGMFJyORkJzQc8Nz+JHiEJNn2tdD0UhwFn3MYWENJVz36JhU7HsvHd9UhBIXVPnYWGzE NGvJRTVLvMsMH4wYv6AWqBeh1xNzOyP7vAZQ/KyohTZqiBoX1U5GT4xkyBmPMlsTXXzE X8kg== X-Gm-Message-State: APjAAAX/qdPVA2CtrTPYWfTnoK1K6+X0KERPal44zkpSXseD5Iz0fUSY B+01TXuIz73i5u1pxeDU/YQKIUd14hSCKw== X-Google-Smtp-Source: APXvYqx8rFrs4AnAhqX3XuKnoGG0oufyG/3Fv7AaHiubO9nTLm5j+kgZ6Ox5HMDM/EvVln/PwjUnlQ== X-Received: by 2002:a17:902:a418:: with SMTP id p24mr17853946plq.46.1575678832954; Fri, 06 Dec 2019 16:33:52 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:52 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 07/13] git-p4: convert path to unicode before processing them Date: Fri, 6 Dec 2019 16:33:26 -0800 Message-Id: <20191207003333.3228-9-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org P4 allows essentially arbitrary encoding for path data while we would perfer to be dealing only with unicode strings. Since path data need to survive round-trip back to p4, this patch implements the general policy that we store path data as-is, but decode them to unicode before doing any non-trivial processing. A new `decode_path()` method is provided that generally does the correct conversion, taking into account `git-p4.pathEncoding` configuration. For python2.7, path strings will be left as-is if it only contains ASCII characters. For python3, decoding is always done so that we have str objects. Signed-off-by: Yang Zhao --- git-p4.py | 67 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 43 insertions(+), 24 deletions(-) diff --git a/git-p4.py b/git-p4.py index bd3118e0e8..650c11eb62 100755 --- a/git-p4.py +++ b/git-p4.py @@ -170,6 +170,21 @@ def decode_text_stream(s): def encode_text_stream(s): return s.encode('utf_8') if isinstance(s, unicode) else s +def decode_path(path): + """Decode a given string (bytes or otherwise) using configured path encoding options + """ + encoding = gitConfig('git-p4.pathEncoding') or 'utf_8' + if bytes is not str: + return path.decode(encoding, errors='replace') if isinstance(path, bytes) else path + else: + try: + path.decode('ascii') + except: + path = path.decode(encoding, errors='replace') + if verbose: + print('Path with non-ASCII characters detected. Used {} to decode: {}'.format(encoding, path)) + return path + def write_pipe(c, stdin): if verbose: sys.stderr.write('Writing pipe: %s\n' % str(c)) @@ -720,7 +735,8 @@ def p4Where(depotPath): if "depotFile" in entry: # Search for the base client side depot path, as long as it starts with the branch's P4 path. # The base path always ends with "/...". - if entry["depotFile"].find(depotPath) == 0 and entry["depotFile"][-4:] == "/...": + entry_path = decode_path(entry['depotFile']) + if entry_path.find(depotPath) == 0 and entry_path[-4:] == "/...": output = entry break elif "data" in entry: @@ -735,11 +751,11 @@ def p4Where(depotPath): return "" clientPath = "" if "path" in output: - clientPath = output.get("path") + clientPath = decode_path(output['path']) elif "data" in output: data = output.get("data") - lastSpace = data.rfind(" ") - clientPath = data[lastSpace + 1:] + lastSpace = data.rfind(b" ") + clientPath = decode_path(data[lastSpace + 1:]) if clientPath.endswith("..."): clientPath = clientPath[:-3] @@ -2516,7 +2532,7 @@ def update_client_spec_path_cache(self, files): """ Caching file paths by "p4 where" batch query """ # List depot file paths exclude that already cached - fileArgs = [f['path'] for f in files if f['path'] not in self.client_spec_path_cache] + fileArgs = [f['path'] for f in files if decode_path(f['path']) not in self.client_spec_path_cache] if len(fileArgs) == 0: return # All files in cache @@ -2531,16 +2547,18 @@ def update_client_spec_path_cache(self, files): if "unmap" in res: # it will list all of them, but only one not unmap-ped continue + depot_path = decode_path(res['depotFile']) if gitConfigBool("core.ignorecase"): - res['depotFile'] = res['depotFile'].lower() - self.client_spec_path_cache[res['depotFile']] = self.convert_client_path(res["clientFile"]) + depot_path = depot_path.lower() + self.client_spec_path_cache[depot_path] = self.convert_client_path(res["clientFile"]) # not found files or unmap files set to "" for depotFile in fileArgs: + depotFile = decode_path(depotFile) if gitConfigBool("core.ignorecase"): depotFile = depotFile.lower() if depotFile not in self.client_spec_path_cache: - self.client_spec_path_cache[depotFile] = "" + self.client_spec_path_cache[depotFile] = b'' def map_in_client(self, depot_path): """Return the relative location in the client where this @@ -2658,7 +2676,7 @@ def isPathWanted(self, path): elif path.lower() == p.lower(): return False for p in self.depotPaths: - if p4PathStartsWith(path, p): + if p4PathStartsWith(path, decode_path(p)): return True return False @@ -2667,7 +2685,7 @@ def extractFilesFromCommit(self, commit, shelved=False, shelved_cl = 0): fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] - found = self.isPathWanted(path) + found = self.isPathWanted(decode_path(path)) if not found: fnum = fnum + 1 continue @@ -2701,7 +2719,7 @@ def stripRepoPath(self, path, prefixes): if self.useClientSpec: # branch detection moves files up a level (the branch name) # from what client spec interpretation gives - path = self.clientSpecDirs.map_in_client(path) + path = decode_path(self.clientSpecDirs.map_in_client(path)) if self.detectBranches: for b in self.knownBranches: if p4PathStartsWith(path, b + "/"): @@ -2735,14 +2753,15 @@ def splitFilesIntoBranches(self, commit): branches = {} fnum = 0 while "depotFile%s" % fnum in commit: - path = commit["depotFile%s" % fnum] + raw_path = commit["depotFile%s" % fnum] + path = decode_path(raw_path) found = self.isPathWanted(path) if not found: fnum = fnum + 1 continue file = {} - file["path"] = path + file["path"] = raw_path file["rev"] = commit["rev%s" % fnum] file["action"] = commit["action%s" % fnum] file["type"] = commit["type%s" % fnum] @@ -2751,7 +2770,7 @@ def splitFilesIntoBranches(self, commit): # start with the full relative path where this file would # go in a p4 client if self.useClientSpec: - relPath = self.clientSpecDirs.map_in_client(path) + relPath = decode_path(self.clientSpecDirs.map_in_client(path)) else: relPath = self.stripRepoPath(path, self.depotPaths) @@ -2789,14 +2808,15 @@ def encodeWithUTF8(self, path): # - helper for streamP4Files def streamOneP4File(self, file, contents): - relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes) - relPath = self.encodeWithUTF8(relPath) + file_path = file['depotFile'] + relPath = self.stripRepoPath(decode_path(file_path), self.branchPrefixes) + if verbose: if 'fileSize' in self.stream_file: size = int(self.stream_file['fileSize']) else: size = 0 # deleted files don't get a fileSize apparently - sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024)) + sys.stdout.write('\r%s --> %s (%i MB)\n' % (file_path, relPath, size/1024/1024)) sys.stdout.flush() (type_base, type_mods) = split_p4_type(file["type"]) @@ -2814,7 +2834,7 @@ def streamOneP4File(self, file, contents): # to nothing. This causes p4 errors when checking out such # a change, and errors here too. Work around it by ignoring # the bad symlink; hopefully a future change fixes it. - print("\nIgnoring empty symlink in %s" % file['depotFile']) + print("\nIgnoring empty symlink in %s" % file_path) return elif data[-1] == '\n': contents = [data[:-1]] @@ -2833,7 +2853,7 @@ def streamOneP4File(self, file, contents): # just the native "NT" type. # try: - text = p4_read_pipe(['print', '-q', '-o', '-', '%s@%s' % (file['depotFile'], file['change'])]) + text = p4_read_pipe(['print', '-q', '-o', '-', '%s@%s' % (decode_path(file['depotFile']), file['change'])], raw=True) except Exception as e: if 'Translation of file content failed' in str(e): type_base = 'binary' @@ -2841,7 +2861,7 @@ def streamOneP4File(self, file, contents): raise e else: if p4_version_string().find('/NT') >= 0: - text = text.replace('\r\n', '\n') + text = text.replace(b'\r\n', b'\n') contents = [ text ] if type_base == "apple": @@ -2872,8 +2892,7 @@ def streamOneP4File(self, file, contents): self.writeToGitStream(git_mode, relPath, contents) def streamOneP4Deletion(self, file): - relPath = self.stripRepoPath(file['path'], self.branchPrefixes) - relPath = self.encodeWithUTF8(relPath) + relPath = self.stripRepoPath(decode_path(file['path']), self.branchPrefixes) if verbose: sys.stdout.write("delete %s\n" % relPath) sys.stdout.flush() @@ -3060,8 +3079,8 @@ def commit(self, details, files, branch, parent = "", allow_empty=False): if self.clientSpecDirs: self.clientSpecDirs.update_client_spec_path_cache(files) - files = [f for f in files - if self.inClientSpec(f['path']) and self.hasBranchPrefix(f['path'])] + files = [f for (f, path) in ((f, decode_path(f['path'])) for f in files) + if self.inClientSpec(path) and self.hasBranchPrefix(path)] if gitConfigBool('git-p4.keepEmptyCommits'): allow_empty = True From patchwork Sat Dec 7 00:33:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B89FD6C1 for ; Sat, 7 Dec 2019 00:34:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 96C2F205ED for ; Sat, 7 Dec 2019 00:34:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="cGFap2aY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726673AbfLGAd7 (ORCPT ); Fri, 6 Dec 2019 19:33:59 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:37420 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726619AbfLGAd4 (ORCPT ); Fri, 6 Dec 2019 19:33:56 -0500 Received: by mail-pg1-f193.google.com with SMTP id q127so4139133pga.4 for ; Fri, 06 Dec 2019 16:33:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SAs02EjjdehEFOiy+U41vm8FPVh9OER+M06Mk6ZqvFk=; b=cGFap2aYKRR7Yf5gQiMc4SuN6kImXVwppE7qiu6NTOuSu9brUFZhfFQe32KAfzFbtp zlvH1+Ac+5uUhvnM3RS/NX9ju6l7W9IAcWhcUDHQY3/u42uJAPLRiCyijzHeq+SOVKuc 16dUyxTQ7zgbwUxmd1DPIPbLBc3f86fi/GVZFOPVz7lqBlnIt/idwiXAb2KoUK2ZonoR opFZg240U+Vv3BSbTFWkxPwD0BMvkjh97DpAbnsUl3fqWWDbsMTnwzt2N9dqzKfS35a0 OxiN7Um5pvWPVzyvXoYHtBafVWcJsTbDXbzNZht5VDEMCLpXkZ6VXdyZZXmVK4/ZTb7C Al1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SAs02EjjdehEFOiy+U41vm8FPVh9OER+M06Mk6ZqvFk=; b=fMd3Bs3JDir1DTVNSaOWDJXKNXVApofRuf7A+GtgrbO1hmJX5ARtCYIDQot1auj5pr ttMen3pYolYgwalh2HnqGq3KPTilvNQyjHkJNd+ZgdAVo2gdYMxUuFsk1ei1zKucD928 idyDIJqRm+f6HS9N4hEAml8qg+N6SfOybfqhSfLeZiDy5q3lb3rylBmYy80LvbO69YbA Xh7E95+SRg9e8ottH+VFbjQtpGijDHkf62BQz4TUp+WAWZTGUaB32oTxG5dFT9ceXOaZ Jbn0kZXC4X0RRNPN26Sb6fxm+zNSRa/PkdxeLBx3rTJ8pTiMbxzGE2oUX5hozkPNWAqJ Bf1w== X-Gm-Message-State: APjAAAUwLMVH8cRJSdonY6ijd88xYX2Gh2Dm1N8Q1gPQaRjGOV7zwOpg 1WqWGzbBvCGOItVUYso5rGTCUGeK/IdrCg== X-Google-Smtp-Source: APXvYqwXiIfIWQmLuKIyXZhNvCQGoEnrYRh8GRpBw1KENimifP+7xWGVGHarDCpN5pXJ6E5mTA/p1w== X-Received: by 2002:a62:f243:: with SMTP id y3mr17684264pfl.146.1575678835720; Fri, 06 Dec 2019 16:33:55 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:54 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 08/13] git-p4: use marshal format version 2 when sending to p4 Date: Fri, 6 Dec 2019 16:33:28 -0800 Message-Id: <20191207003333.3228-11-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org p4 does not appear to understand marshal format version 3 and above. Version 2 was the latest supported by python-2.7. Signed-off-by: Yang Zhao --- git-p4.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index af563cf23d..c2a3de59e7 100755 --- a/git-p4.py +++ b/git-p4.py @@ -1697,7 +1697,8 @@ def modifyChangelistUser(self, changelist, newUser): c = changes[0] if c['User'] == newUser: return # nothing to do c['User'] = newUser - input = marshal.dumps(c) + # p4 does not understand format version 3 and above + input = marshal.dumps(c, 2) result = p4CmdList("change -f -i", stdin=input) for r in result: From patchwork Sat Dec 7 00:33:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277231 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32F7914B7 for ; Sat, 7 Dec 2019 00:34:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0C6322245C for ; Sat, 7 Dec 2019 00:34:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="MIlkw5xm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726653AbfLGAd6 (ORCPT ); Fri, 6 Dec 2019 19:33:58 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:35836 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726642AbfLGAd5 (ORCPT ); Fri, 6 Dec 2019 19:33:57 -0500 Received: by mail-pf1-f194.google.com with SMTP id b19so4202144pfo.2 for ; Fri, 06 Dec 2019 16:33:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RqltTGL002GBLQBDqp/dy8Ld/1ou8CKL0hBJ5HgFgtY=; b=MIlkw5xmgDgKVHdkbHA1UFrXU5278nH0jwB8J5uZjMykGXIQn5wo9zWhyxVCUpCBX0 BRreDcyf1qWofIfBJ8WCiCXsp2QlhILMj4Omhdwtalj0fvkPHnrdoU1Ddvti4PtkhMJt 1vXlyKScxDtKPUn6FS3n7hm4GIsdguDBNX9x5e7TXKZgf292As2s1jnXMVnFxZwnXSPN AqzMG+uqgPcnvYQBnvWyaPL2mU+icFlIiegcK7Prug2ykfYKMBZ7SH9Ptz4Znb1mgHJC NNho9+7ugyDYflSPuZ6GMtLljLP+pINBTPI5SpgWZZe4xDvbgFWQVfi+iTx/87hHMvXN Eozg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RqltTGL002GBLQBDqp/dy8Ld/1ou8CKL0hBJ5HgFgtY=; b=BB2Zsn5Gp6IfTF/FUlHayLbHOYckTFmVeATgDrf3j+L2DzMWuksti2YnKjX/CUyRiK 32CYk14OrRoBzCUFuc5lzDNBgR3Rax/527fu3fuHyCV4+cm3TmBg4YGF05Rrl0CNDnaL irHWIwBqmQBjh3PTqtejXwXXUeanMEqvt1oL1FNwh2HnfropAZ7KgKpLwH/CK+QpWWOB gAjrM3kzeJoM4HjToVX907MipAOIRil1aKzXi1qHoPrJC42e5+x+HamKQXZa8dzz9law qleFPb9kB5j7suvR/tY9l8PU18JNLtxv86xvmnO0hA5f19Kcyp7VLirAjlYd4sHhGt/B 9xpg== X-Gm-Message-State: APjAAAX3AJlXE2vOJesfYW2rXW6W+Qi45Y2q9GuX/Nk2IqJjBuu9a4aq 8aPRzXTstw1Q+RZsEMXtmigVLESbmx3ftQ== X-Google-Smtp-Source: APXvYqxzXGISeQxlVwCG2suVF8J0GDUDgnlxOvCz2lBG86JF18uG18oRbS5OkAJuxif/zNZq06MyUQ== X-Received: by 2002:aa7:85d8:: with SMTP id z24mr18046547pfn.202.1575678836812; Fri, 06 Dec 2019 16:33:56 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:56 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 09/13] git-p4: fix freezing while waiting for fast-import progress Date: Fri, 6 Dec 2019 16:33:29 -0800 Message-Id: <20191207003333.3228-12-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As part of its importing process, git-p4 sends a `checkpoint` followed immediately by `progress` to fast-import in to force synchronization. Due to buffering, it is possible for the `progress` command to not be flushed before git-p4 proceeds to wait for the corresponding response. This causes the script to freeze completely, and is consistently observable at least on python-3.6.9. Make sure this command sequence is completely flushed before waiting. Signed-off-by: Yang Zhao --- git-p4.py | 1 + 1 file changed, 1 insertion(+) diff --git a/git-p4.py b/git-p4.py index c2a3de59e7..1007b936c8 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2659,6 +2659,7 @@ def __init__(self): def checkpoint(self): self.gitStream.write("checkpoint\n\n") self.gitStream.write("progress checkpoint\n\n") + self.gitStream.flush() out = self.gitOutput.readline() if self.verbose: print("checkpoint finished: " + out) From patchwork Sat Dec 7 00:33:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68350109A for ; Sat, 7 Dec 2019 00:34:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4444721835 for ; Sat, 7 Dec 2019 00:34:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="nwuJ1I49" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726683AbfLGAeA (ORCPT ); Fri, 6 Dec 2019 19:34:00 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:42476 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726642AbfLGAd7 (ORCPT ); Fri, 6 Dec 2019 19:33:59 -0500 Received: by mail-pg1-f195.google.com with SMTP id i5so4130638pgj.9 for ; Fri, 06 Dec 2019 16:33:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R/vHReJiSdSY85lqadR7LxXCOR/zXJ/7mqcdAb++m80=; b=nwuJ1I49wIf6moyRQEvircAtVQX3cJYKWaJo/L0/XfVMBKoN6sMZUSJnzoZPn5PtOO 3BdRkVoT8H6qtOTUoP49ndDKbh+F0u4SXvrpSe0YWA1FbxIGPEGX991i3zvsY1qIURVR YV4RpR1WNxwf/u2DWTFr1kpXyb84uFXGVLMUzL7UrlO8K/6wxeNIMAMsAysz2LIy73sc iS5ZdAWQV4yarqP2T2vJ4M0OU5wqEst+J8L+Pn9tkhtNd98jLpFw6uPTACf5kP19BzCE kcKENB78Pny0yRKyDBZPt07KTTZ8jL3uY1hlmdwxid0NV+omJiND5LVxsqhfJbuRE4Zj NYgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R/vHReJiSdSY85lqadR7LxXCOR/zXJ/7mqcdAb++m80=; b=c0pHIazo9e27YQ+ezPbiXNMuYxipRB+L42o9l4buhp6EmgkXjMVzfS1zSNxYvkksMx swKaGJPhKW42XcstYmoio9HvDgtdZfQsqoopsfDwwihioUX51LXLp1vAWph9uTy9lu5Z lfqn9qX8OlLVtbmKpIS7bkn1/7supOv4L+F+J9qJgYZTxoQoUW4WJ7WDaww7W+PdMdIZ ijhMHkhvQi5pZ76X+Ry5d2BH5jhHJrvnTrqj1QUaumwKCtwBNyIGWMFBTh16tCqDiTUB ioGt67oq4oD2rPPZ8zmpAExJAzL/8NlJ4WOBzwSPDyn1EUtb/Bj9DZ2rMKucUqUt7QqO z+fg== X-Gm-Message-State: APjAAAWY2jydfuWWFNemcFg74S1Lp6IW1b/wvqSTkUQX7saPw5ZwoD9m iq+r2We6iXJO74W3XFKLO6WDfkfZ9Ig6Mg== X-Google-Smtp-Source: APXvYqwMRdrMHqVfJxu51Er79IitdE32d/iVa29NJnyw9ztPV28xGr+IMp8ZTKqEjw9LOfutOOMkkQ== X-Received: by 2002:a62:1687:: with SMTP id 129mr17684638pfw.44.1575678838322; Fri, 06 Dec 2019 16:33:58 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:57 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 10/13] git-p4: use functools.reduce instead of reduce Date: Fri, 6 Dec 2019 16:33:30 -0800 Message-Id: <20191207003333.3228-13-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org For python3, reduce() has been moved to functools.reduce(). This is also available in python2.7. Signed-off-by: Yang Zhao --- git-p4.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index 1007b936c8..c888e4825a 100755 --- a/git-p4.py +++ b/git-p4.py @@ -13,6 +13,7 @@ sys.exit(1) import os import optparse +import functools import marshal import subprocess import tempfile @@ -1176,7 +1177,7 @@ def pushFile(self, localLargeFile): assert False, "Method 'pushFile' required in " + self.__class__.__name__ def hasLargeFileExtension(self, relPath): - return reduce( + return functools.reduce( lambda a, b: a or b, [relPath.endswith('.' + e) for e in gitConfigList('git-p4.largeFileExtensions')], False From patchwork Sat Dec 7 00:33:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73FCF109A for ; Sat, 7 Dec 2019 00:34:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4E48021835 for ; Sat, 7 Dec 2019 00:34:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="BTlxSQyj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726697AbfLGAeB (ORCPT ); Fri, 6 Dec 2019 19:34:01 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:46939 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfLGAeA (ORCPT ); Fri, 6 Dec 2019 19:34:00 -0500 Received: by mail-pg1-f196.google.com with SMTP id z124so4119736pgb.13 for ; Fri, 06 Dec 2019 16:33:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mOA+CO7BKBTw54HSlvaLhxTmlUJ7yNCrhw0TQw7OcYw=; b=BTlxSQyjZu4xNDzp9Aoy/qpPTmHXSOYcgNsat9cBnfZNvwgtcmCXJ6W3qxa0WGFWBF 5jv7pK+HaFU5SiL39P2qwygy7p3IAtESMx5qHmFBikHhEdFTXLFKf1QflGVdFxWVsrzL f/fdsCcOzT8oqpWVeATjVoB6U9RhP+qh3qBdutCR5B60C3dJlR7YD/mB0k72urbKb8lw 8OKJ80cFlUS92Cu2roOF8X83Gr1KhlgjkkMZr0QpVcnPHP4wGX4e1sMbDyU5QEBynq0m Ab9gBh00yt5Oay99kwmNPWQVdVn+AYZlRTvX1MH/bXwBE6lXg4EdqQU3oN0fhoXU4KGR Nk7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mOA+CO7BKBTw54HSlvaLhxTmlUJ7yNCrhw0TQw7OcYw=; b=tiJl0ISILIXyEYNJg6jpGr4Z0In+6a4O1l5lR88wmUC+pRD0jwj3QFD3VgwZYGbnyO p8SKNFxCMCY3DhQsE92Ho645Y0mLGCNa2jG/29/pDWv9ilHS3YjhnAb8DqHh2DpmlpCC hjuQdBvTKqoayVPMdJVx/G6qYwV8SzxBtC9WTlaHdJ87lxcjGF+s/MSHqf6Eqbz+ARI5 C2l52XEuArhVjgGWUYcYkp7Jpm6doMn7RhT0mvGOYOpXLqmBMykXIt5sid1WMrlt35da vdb1PqBrBGn6A0OTwyVQDTMaTYaa81WVd6WrWY44e3znsYixiRgUT09nW1j8m7iatbYu +uMw== X-Gm-Message-State: APjAAAXTsi7BmFta/gfX+BTuHuu7L9pzZaAxDER9fjVewaGWG4+0Pcld dfcAn4D6mWiNl+dxzMOTCgydWBp4qNkXTA== X-Google-Smtp-Source: APXvYqyFpnueDGC313WLLiCHxGK7PEOcu5GOw/nMiy9A5ZcpAAq9eekl/PFHnI2tAN9JcHolnLuEMg== X-Received: by 2002:a63:4f64:: with SMTP id p36mr6855102pgl.271.1575678839244; Fri, 06 Dec 2019 16:33:59 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:58 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 11/13] git-p4: use dict.items() iteration for python3 compatibility Date: Fri, 6 Dec 2019 16:33:31 -0800 Message-Id: <20191207003333.3228-14-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Python3 uses dict.items() instead of .iteritems() to provide iteratoration over dict. Although items() is technically less efficient for python2.7 (allocates a new list instead of simply iterating), the amount of data involved is very small and the penalty negligible. Signed-off-by: Yang Zhao --- git-p4.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index c888e4825a..867a8d42ef 100755 --- a/git-p4.py +++ b/git-p4.py @@ -1763,7 +1763,7 @@ def prepareSubmitTemplate(self, changelist=None): break if not change_entry: die('Failed to decode output of p4 change -o') - for key, value in change_entry.iteritems(): + for key, value in change_entry.items(): if key.startswith('File'): if 'depot-paths' in settings: if not [p for p in settings['depot-paths'] From patchwork Sat Dec 7 00:33:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B10C86C1 for ; Sat, 7 Dec 2019 00:34:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8979521835 for ; Sat, 7 Dec 2019 00:34:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="Uuoqp20c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726706AbfLGAeE (ORCPT ); Fri, 6 Dec 2019 19:34:04 -0500 Received: from mail-pj1-f68.google.com ([209.85.216.68]:43723 "EHLO mail-pj1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726689AbfLGAeB (ORCPT ); Fri, 6 Dec 2019 19:34:01 -0500 Received: by mail-pj1-f68.google.com with SMTP id g4so3425728pjs.10 for ; Fri, 06 Dec 2019 16:34:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0cPvVRg3ltl7mKdxuUrHifmSFOZ7d/OAdqtpr6njhWI=; b=Uuoqp20chA+cM1ffMs63ZuGyIfhDj0T3WHqMS30sLuj//6ioOsUpmUbVH948cp0De1 FDtZpwSvSaEsUKg2OHXsjtRlbuCJXMomUe5T0eRJOtr/NIWgajUCZZZux3i64Zm12KTY PcEH9kONHFnGsJILNh8lxPSHHv9Kf0InVXXGzqOUJaZJUOIcfY8NvlyfXobPyUB9ykCg cUAzVPxjxck3GCRhaqVi70Y+31738NqVWbnlUeq1uMgjLM0St234dOjcqJn1meqD4LhR SDljlyq8q+zzGbgo8/7jrz1i2NRYZSKtjU3sVp+xuMyjLBryJQbM3nxVw0QsgE6rAzm/ VC6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0cPvVRg3ltl7mKdxuUrHifmSFOZ7d/OAdqtpr6njhWI=; b=U/yJsWaHLnOfpFiQbdGGvgzNjpOYDkt4ZiYBiUzppUbS+HC3CqmLHouhKacfv+u3Is rq/FvAzuON6Xk0fx1oHcM8mD2eSkAD6T3lldZYzJlAxikOUBv+vKM03hYvxUu6YAGLIf OEzZLRRfo4eyVmk0u7b14z3g13kE8Qo++r4eQWrNOAfoPdz7mweWiR3tR3gBeXEByoeo azfV4ms7yjiBpYFhuh9rtRBq+/0EpWEAm0y36cRaeYV3d4S4du4W5Vgo97BHDx6hOeBt h9K2rDlJfnQpYHi55rPKQq8DeT6Cluep6LbwBqGX4nRbKc+LPKzV3agPNUYt6T+eyWy4 nxtg== X-Gm-Message-State: APjAAAWmKs4QDnRp9EkzdfEUhe+pLiGefQoA8u2RMNTzRt/earp8YWAz E5BEcZIJU/CJ90rGGyBYCFTjTk5kK1nTFw== X-Google-Smtp-Source: APXvYqxaYkJ5mqRuWq9GiB6FiKvJlPYOMIkN0tuNXw+uXehHOnxrD48+BM+JgMyCKofP6HcKJM+ZWA== X-Received: by 2002:a17:90a:b318:: with SMTP id d24mr17203179pjr.142.1575678840306; Fri, 06 Dec 2019 16:34:00 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.33.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:33:59 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 12/13] git-p4: simplify regex pattern generation for parsing diff-tree Date: Fri, 6 Dec 2019 16:33:32 -0800 Message-Id: <20191207003333.3228-15-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It is not clear why a generator was used to create the regex used to parse git-diff-tree output; I assume an early implementation required it, but is not part of the mainline change. Simply use a lazily initialized global instead. Signed-off-by: Yang Zhao --- git-p4.py | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/git-p4.py b/git-p4.py index 867a8d42ef..f975f197a5 100755 --- a/git-p4.py +++ b/git-p4.py @@ -562,12 +562,7 @@ def getGitTags(): gitTags.add(tag) return gitTags -def diffTreePattern(): - # This is a simple generator for the diff tree regex pattern. This could be - # a class variable if this and parseDiffTreeEntry were a part of a class. - pattern = re.compile(':(\d+) (\d+) (\w+) (\w+) ([A-Z])(\d+)?\t(.*?)((\t(.*))|$)') - while True: - yield pattern +_diff_tree_pattern = None def parseDiffTreeEntry(entry): """Parses a single diff tree entry into its component elements. @@ -588,7 +583,11 @@ def parseDiffTreeEntry(entry): If the pattern is not matched, None is returned.""" - match = diffTreePattern().next().match(entry) + global _diff_tree_pattern + if not _diff_tree_pattern: + _diff_tree_pattern = re.compile(':(\d+) (\d+) (\w+) (\w+) ([A-Z])(\d+)?\t(.*?)((\t(.*))|$)') + + match = _diff_tree_pattern.match(entry) if match: return { 'src_mode': match.group(1), From patchwork Sat Dec 7 00:33:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Zhao X-Patchwork-Id: 11277239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A431D6C1 for ; Sat, 7 Dec 2019 00:34:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7811F2245C for ; Sat, 7 Dec 2019 00:34:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=skyboxlabs-com.20150623.gappssmtp.com header.i=@skyboxlabs-com.20150623.gappssmtp.com header.b="FhnICS6A" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726717AbfLGAeE (ORCPT ); Fri, 6 Dec 2019 19:34:04 -0500 Received: from mail-pf1-f173.google.com ([209.85.210.173]:41038 "EHLO mail-pf1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726678AbfLGAeC (ORCPT ); Fri, 6 Dec 2019 19:34:02 -0500 Received: by mail-pf1-f173.google.com with SMTP id s18so4184842pfd.8 for ; Fri, 06 Dec 2019 16:34:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skyboxlabs-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1zhXpLzZDgtm1MlyEi+R/FJfQEttb6Pu04P7Nh/tH+A=; b=FhnICS6A88M1W1usD79l85NFAGGC3VFYjzOUq52m6ECWzzJVvJESHrFRLWA6jmZl88 3SC+/gUFcBGBdHVFHsveSdDCCVPLxe3PZlfpjADBa58qZnEImS7xbmJy1sPCPrPc/a8C 2DXOCf2F+zW22qAChxgb3i0b14MX7tiXUNkXXRc42G6/BvCHmfIQSSiouyQe+t3L+iEG zZK9rF9wAgZzCLeT1gL6YdCeylenswhR4x8bjiT4pe2mUcTzhS5KCDgnAlQt+c4gm9jn xwoyqr+lD2+Y0pEcKEHlJW47TgsIeavGtU1GKQWI+Czj9jKyPQ0SJvnNE2NMBt417fu0 ySVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1zhXpLzZDgtm1MlyEi+R/FJfQEttb6Pu04P7Nh/tH+A=; b=Eq4CWTFTnY4IpRjg1BHAUY+DD1PXIMH5vS0Hg0878Y+9ODXGn9PCeGWAHSOTuqTVfV uZj0s+X+qOQNhMBz/SYyezU8u2pt9FYWV6yL2llcCylmthKk1S8RiIJosNl5vwq+1rjd flWgdRDPe72H6Q6foCcZitdkXDcwr6HHupvZTp87iILcqcro4cXraaFEhGkN7JWl8m3O tlPhPjnfQkQ3XRfIcWmrvSxAN5EM/6aNi3r4f5F3fgjTzZck860gvi2H9a2rl7Vs6PkR XoWlf/ehVgHbaIwcTQ458RpDxMFqOIIYgPFLxO+7AlQBiT6YEXCl++nmB3Xi5qrfB+XH 9J2g== X-Gm-Message-State: APjAAAW/wpHHG0ZT4LfOxZuQoUHK6Buz1c0BZwko5lSfciZV1keUfvI3 Z68iCuouV/OU/aIchzFEgWkaf3h9YNhzgg== X-Google-Smtp-Source: APXvYqxUbWrLOWnyq4NGCNMuK2Kms/BCOMT8NJDO7yq4RB4rK6zFkS6sH07LMP8xYdmvo50TTLOSgQ== X-Received: by 2002:a63:c406:: with SMTP id h6mr6756530pgd.213.1575678841276; Fri, 06 Dec 2019 16:34:01 -0800 (PST) Received: from SBL-LP-YZHAO.skyboxlabs.local (d173-180-108-168.bchsia.telus.net. [173.180.108.168]) by smtp.gmail.com with ESMTPSA id b21sm18708949pfp.0.2019.12.06.16.34.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Dec 2019 16:34:00 -0800 (PST) From: Yang Zhao To: git@vger.kernel.org Cc: Yang Zhao Subject: [PATCH 13/13] git-p4: use python3's input() everywhere Date: Fri, 6 Dec 2019 16:33:33 -0800 Message-Id: <20191207003333.3228-16-yang.zhao@skyboxlabs.com> X-Mailer: git-send-email 2.21.0.windows.1 In-Reply-To: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> References: <20191207003333.3228-1-yang.zhao@skyboxlabs.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Python3 deprecates raw_input() from 2.7 and replaced it with input(). Since we do not need 2.7's input() semantics, `raw_input()` is aliased to `input()` for easy forward compatability. Signed-off-by: Yang Zhao --- git-p4.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/git-p4.py b/git-p4.py index f975f197a5..97a9def657 100755 --- a/git-p4.py +++ b/git-p4.py @@ -33,6 +33,8 @@ else: bytes = str basestring = basestring + # We want python3's input() semantics + input = raw_input try: from subprocess import CalledProcessError @@ -1819,7 +1821,7 @@ def edit_template(self, template_file): return True while True: - response = raw_input("Submit template unchanged. Submit anyway? [y]es, [n]o (skip this patch) ") + response = input("Submit template unchanged. Submit anyway? [y]es, [n]o (skip this patch) ") if response == 'y': return True if response == 'n': @@ -2390,7 +2392,7 @@ def run(self, args): # prompt for what to do, or use the option/variable if self.conflict_behavior == "ask": print("What do you want to do?") - response = raw_input("[s]kip this commit but apply" + response = input("[s]kip this commit but apply" " the rest, or [q]uit? ") if not response: continue