From patchwork Wed Jun 5 23:24:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 10978025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43BA313AD for ; Wed, 5 Jun 2019 23:24:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 361E2283C3 for ; Wed, 5 Jun 2019 23:24:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2AA2E286D0; Wed, 5 Jun 2019 23:24:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2DEC828A19 for ; Wed, 5 Jun 2019 23:24:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726653AbfFEXYm (ORCPT ); Wed, 5 Jun 2019 19:24:42 -0400 Received: from mail-eopbgr30054.outbound.protection.outlook.com ([40.107.3.54]:39078 "EHLO EUR03-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726606AbfFEXYm (ORCPT ); Wed, 5 Jun 2019 19:24:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FYPg42gtesq0bzoKy8Q694OjJSFnWggRTBx/aw2TKsA=; b=G26lN5fs40nHxntl0WAUPr/9j6iNpjaJbPpWdNBBcQq5OcGK9Dp6lgS3dan2jfF8eRKEpq1TsOSv2By83mc33fe4/uJlsVgd1HMutmMRWiWavZYXWHQ54Tziw8K8R4VLM2/MFKybR4/JcUQmP15Bs9f3Kbo0Ec/aOC4YzcItLTk= Received: from DB8PR05MB5898.eurprd05.prod.outlook.com (20.179.9.32) by DB8PR05MB6105.eurprd05.prod.outlook.com (20.179.10.223) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1943.22; Wed, 5 Jun 2019 23:24:34 +0000 Received: from DB8PR05MB5898.eurprd05.prod.outlook.com ([fe80::4008:6417:32d4:6031]) by DB8PR05MB5898.eurprd05.prod.outlook.com ([fe80::4008:6417:32d4:6031%5]) with mapi id 15.20.1965.011; Wed, 5 Jun 2019 23:24:34 +0000 From: Saeed Mahameed To: "David S. Miller" , Jason Gunthorpe , Doug Ledford CC: Michael Chan , Andy Gospodarek , Tal Gilboa , "linux-rdma@vger.kernel.org" , "netdev@vger.kernel.org" , Saeed Mahameed Subject: [for-next 1/9] linux/dim: Move logic to dim.h Thread-Topic: [for-next 1/9] linux/dim: Move logic to dim.h Thread-Index: AQHVG/XVnjam2T9Um0qtthoHLiQ/BA== Date: Wed, 5 Jun 2019 23:24:34 +0000 Message-ID: <20190605232348.6452-2-saeedm@mellanox.com> References: <20190605232348.6452-1-saeedm@mellanox.com> In-Reply-To: <20190605232348.6452-1-saeedm@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-originating-ip: [209.116.155.178] x-clientproxiedby: BYAPR02CA0045.namprd02.prod.outlook.com (2603:10b6:a03:54::22) To DB8PR05MB5898.eurprd05.prod.outlook.com (2603:10a6:10:a4::32) authentication-results: spf=none (sender IP is ) smtp.mailfrom=saeedm@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 48ff5696-ae6e-48f5-52e9-08d6ea0cf7d6 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:DB8PR05MB6105; x-ms-traffictypediagnostic: DB8PR05MB6105: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:262; x-forefront-prvs: 00594E8DBA x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(396003)(346002)(39850400004)(136003)(366004)(376002)(199004)(189003)(4326008)(316002)(5660300002)(2906002)(66946007)(53936002)(186003)(8936002)(66476007)(8676002)(81166006)(66446008)(1076003)(478600001)(86362001)(50226002)(81156014)(66556008)(14454004)(107886003)(6116002)(73956011)(7736002)(305945005)(446003)(11346002)(64756008)(3846002)(71190400001)(99286004)(26005)(25786009)(110136005)(71200400001)(486006)(2616005)(68736007)(6512007)(66066001)(6436002)(76176011)(6486002)(54906003)(36756003)(6506007)(386003)(102836004)(256004)(476003)(52116002);DIR:OUT;SFP:1101;SCL:1;SRVR:DB8PR05MB6105;H:DB8PR05MB5898.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Z7NcfllSrHZ0CCTB9KqL+HH8APxKewv34e55IeZuSigPGK0dXcsEQFHTYXDNGqm7ys6aDdwGHUCAorECyYeEgMD0iMDXyuxrNgYfqKQferdn3IVr9A2LdyISQ/L5AnGR4QXtgBtViHQfY6cq/Z6m1YNVhQUSf505IA0yqrOE+ujH2LFw3HzjLR/6cdC9x7y7IJUkbuJMjnAFmJsIBH2poC58gFkC+ANiugk7jSMZmr+Nn+AEN4kd3RhLRp1Jqxt0rd+nOmn+59TNcZ2ET/HzsGZDwEL/gR5wY9IH/G9C/pzeitzcM+++atCtXc7n12faoGlP3qLSZG7rMV9tmu0Im8o9Hn9BxmSPP4M6v3PWhckI2/jdBgp6Xs2xUtkmbxkNWvoozAa7BCHzbNkczZdcx3q1QoXzO8TInqXD5/jbWVc= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48ff5696-ae6e-48f5-52e9-08d6ea0cf7d6 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jun 2019 23:24:34.1977 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: saeedm@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR05MB6105 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tal Gilboa In preparation for supporting more implementations of the DIM algorithm, I'm moving what would become common logic to a common library. Downstream DIM implementations will use the common lib for their implementation. Signed-off-by: Tal Gilboa Signed-off-by: Saeed Mahameed --- MAINTAINERS | 1 + include/linux/dim.h | 153 ++++++++++++++++++++++++++++++++++++++++ include/linux/net_dim.h | 148 +------------------------------------- 3 files changed, 156 insertions(+), 146 deletions(-) create mode 100644 include/linux/dim.h diff --git a/MAINTAINERS b/MAINTAINERS index 429c6c624861..5d4b852d9d39 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5589,6 +5589,7 @@ DYNAMIC INTERRUPT MODERATION M: Tal Gilboa S: Maintained F: include/linux/net_dim.h +F: include/linux/dim.h DZ DECSTATION DZ11 SERIAL DRIVER M: "Maciej W. Rozycki" diff --git a/include/linux/dim.h b/include/linux/dim.h new file mode 100644 index 000000000000..67d7ca40f3dd --- /dev/null +++ b/include/linux/dim.h @@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2019 Mellanox Technologies. */ + +#ifndef DIM_H +#define DIM_H + +#include + +#define NET_DIM_NEVENTS 64 + +/* more than 10% difference */ +#define IS_SIGNIFICANT_DIFF(val, ref) \ + (((100UL * abs((val) - (ref))) / (ref)) > 10) +#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) \ +& (BIT_ULL(bits) - 1)) + +struct net_dim_cq_moder { + u16 usec; + u16 pkts; + u8 cq_period_mode; +}; + +struct net_dim_sample { + ktime_t time; + u32 pkt_ctr; + u32 byte_ctr; + u16 event_ctr; +}; + +struct net_dim_stats { + int ppms; /* packets per msec */ + int bpms; /* bytes per msec */ + int epms; /* events per msec */ +}; + +struct net_dim { /* Dynamic Interrupt Moderation */ + u8 state; + struct net_dim_stats prev_stats; + struct net_dim_sample start_sample; + struct work_struct work; + u8 profile_ix; + u8 mode; + u8 tune_state; + u8 steps_right; + u8 steps_left; + u8 tired; +}; + +enum { + NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0, + NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1, + NET_DIM_CQ_PERIOD_NUM_MODES +}; + +enum { + NET_DIM_START_MEASURE, + NET_DIM_MEASURE_IN_PROGRESS, + NET_DIM_APPLY_NEW_PROFILE, +}; + +enum { + NET_DIM_PARKING_ON_TOP, + NET_DIM_PARKING_TIRED, + NET_DIM_GOING_RIGHT, + NET_DIM_GOING_LEFT, +}; + +enum { + NET_DIM_STATS_WORSE, + NET_DIM_STATS_SAME, + NET_DIM_STATS_BETTER, +}; + +enum { + NET_DIM_STEPPED, + NET_DIM_TOO_TIRED, + NET_DIM_ON_EDGE, +}; + +static inline bool net_dim_on_top(struct net_dim *net_dim) +{ + switch (net_dim->tune_state) { + case NET_DIM_PARKING_ON_TOP: + case NET_DIM_PARKING_TIRED: + return true; + case NET_DIM_GOING_RIGHT: + return (net_dim->steps_left > 1) && (net_dim->steps_right == 1); + default: /* NET_DIM_GOING_LEFT */ + return (net_dim->steps_right > 1) && (net_dim->steps_left == 1); + } +} + +static inline void net_dim_turn(struct net_dim *net_dim) +{ + switch (net_dim->tune_state) { + case NET_DIM_PARKING_ON_TOP: + case NET_DIM_PARKING_TIRED: + break; + case NET_DIM_GOING_RIGHT: + net_dim->tune_state = NET_DIM_GOING_LEFT; + net_dim->steps_left = 0; + break; + case NET_DIM_GOING_LEFT: + net_dim->tune_state = NET_DIM_GOING_RIGHT; + net_dim->steps_right = 0; + break; + } +} + +static inline void net_dim_park_on_top(struct net_dim *net_dim) +{ + net_dim->steps_right = 0; + net_dim->steps_left = 0; + net_dim->tired = 0; + net_dim->tune_state = NET_DIM_PARKING_ON_TOP; +} + +static inline void net_dim_park_tired(struct net_dim *net_dim) +{ + net_dim->steps_right = 0; + net_dim->steps_left = 0; + net_dim->tune_state = NET_DIM_PARKING_TIRED; +} + +static inline void +net_dim_sample(u16 event_ctr, u64 packets, u64 bytes, struct net_dim_sample *s) +{ + s->time = ktime_get(); + s->pkt_ctr = packets; + s->byte_ctr = bytes; + s->event_ctr = event_ctr; +} + +static inline void +net_dim_calc_stats(struct net_dim_sample *start, struct net_dim_sample *end, + struct net_dim_stats *curr_stats) +{ + /* u32 holds up to 71 minutes, should be enough */ + u32 delta_us = ktime_us_delta(end->time, start->time); + u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr); + u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr, + start->byte_ctr); + + if (!delta_us) + return; + + curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us); + curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us); + curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC, + delta_us); +} + +#endif /* DIM_H */ diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h index fd458389f7d1..373cda74b167 100644 --- a/include/linux/net_dim.h +++ b/include/linux/net_dim.h @@ -35,73 +35,10 @@ #define NET_DIM_H #include - -struct net_dim_cq_moder { - u16 usec; - u16 pkts; - u8 cq_period_mode; -}; - -struct net_dim_sample { - ktime_t time; - u32 pkt_ctr; - u32 byte_ctr; - u16 event_ctr; -}; - -struct net_dim_stats { - int ppms; /* packets per msec */ - int bpms; /* bytes per msec */ - int epms; /* events per msec */ -}; - -struct net_dim { /* Adaptive Moderation */ - u8 state; - struct net_dim_stats prev_stats; - struct net_dim_sample start_sample; - struct work_struct work; - u8 profile_ix; - u8 mode; - u8 tune_state; - u8 steps_right; - u8 steps_left; - u8 tired; -}; - -enum { - NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0, - NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1, - NET_DIM_CQ_PERIOD_NUM_MODES -}; - -/* Adaptive moderation logic */ -enum { - NET_DIM_START_MEASURE, - NET_DIM_MEASURE_IN_PROGRESS, - NET_DIM_APPLY_NEW_PROFILE, -}; - -enum { - NET_DIM_PARKING_ON_TOP, - NET_DIM_PARKING_TIRED, - NET_DIM_GOING_RIGHT, - NET_DIM_GOING_LEFT, -}; - -enum { - NET_DIM_STATS_WORSE, - NET_DIM_STATS_SAME, - NET_DIM_STATS_BETTER, -}; - -enum { - NET_DIM_STEPPED, - NET_DIM_TOO_TIRED, - NET_DIM_ON_EDGE, -}; +#include #define NET_DIM_PARAMS_NUM_PROFILES 5 -/* Adaptive moderation profiles */ +/* Netdev dynamic interrupt moderation profiles */ #define NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE 256 #define NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE 128 #define NET_DIM_DEF_PROFILE_CQE 1 @@ -188,36 +125,6 @@ net_dim_get_def_tx_moderation(u8 cq_period_mode) return net_dim_get_tx_moderation(cq_period_mode, profile_ix); } -static inline bool net_dim_on_top(struct net_dim *dim) -{ - switch (dim->tune_state) { - case NET_DIM_PARKING_ON_TOP: - case NET_DIM_PARKING_TIRED: - return true; - case NET_DIM_GOING_RIGHT: - return (dim->steps_left > 1) && (dim->steps_right == 1); - default: /* NET_DIM_GOING_LEFT */ - return (dim->steps_right > 1) && (dim->steps_left == 1); - } -} - -static inline void net_dim_turn(struct net_dim *dim) -{ - switch (dim->tune_state) { - case NET_DIM_PARKING_ON_TOP: - case NET_DIM_PARKING_TIRED: - break; - case NET_DIM_GOING_RIGHT: - dim->tune_state = NET_DIM_GOING_LEFT; - dim->steps_left = 0; - break; - case NET_DIM_GOING_LEFT: - dim->tune_state = NET_DIM_GOING_RIGHT; - dim->steps_right = 0; - break; - } -} - static inline int net_dim_step(struct net_dim *dim) { if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2)) @@ -245,21 +152,6 @@ static inline int net_dim_step(struct net_dim *dim) return NET_DIM_STEPPED; } -static inline void net_dim_park_on_top(struct net_dim *dim) -{ - dim->steps_right = 0; - dim->steps_left = 0; - dim->tired = 0; - dim->tune_state = NET_DIM_PARKING_ON_TOP; -} - -static inline void net_dim_park_tired(struct net_dim *dim) -{ - dim->steps_right = 0; - dim->steps_left = 0; - dim->tune_state = NET_DIM_PARKING_TIRED; -} - static inline void net_dim_exit_parking(struct net_dim *dim) { dim->tune_state = dim->profile_ix ? NET_DIM_GOING_LEFT : @@ -267,9 +159,6 @@ static inline void net_dim_exit_parking(struct net_dim *dim) net_dim_step(dim); } -#define IS_SIGNIFICANT_DIFF(val, ref) \ - (((100UL * abs((val) - (ref))) / (ref)) > 10) /* more than 10% difference */ - static inline int net_dim_stats_compare(struct net_dim_stats *curr, struct net_dim_stats *prev) { @@ -351,39 +240,6 @@ static inline bool net_dim_decision(struct net_dim_stats *curr_stats, return dim->profile_ix != prev_ix; } -static inline void net_dim_sample(u16 event_ctr, - u64 packets, - u64 bytes, - struct net_dim_sample *s) -{ - s->time = ktime_get(); - s->pkt_ctr = packets; - s->byte_ctr = bytes; - s->event_ctr = event_ctr; -} - -#define NET_DIM_NEVENTS 64 -#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1)) - -static inline void net_dim_calc_stats(struct net_dim_sample *start, - struct net_dim_sample *end, - struct net_dim_stats *curr_stats) -{ - /* u32 holds up to 71 minutes, should be enough */ - u32 delta_us = ktime_us_delta(end->time, start->time); - u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr); - u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr, - start->byte_ctr); - - if (!delta_us) - return; - - curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us); - curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us); - curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC, - delta_us); -} - static inline void net_dim(struct net_dim *dim, struct net_dim_sample end_sample) {