From patchwork Tue Jun 25 20:57:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 11016391 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2FB9A13B4 for ; Tue, 25 Jun 2019 20:57:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 206B626E54 for ; Tue, 25 Jun 2019 20:57:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 14E42284D4; Tue, 25 Jun 2019 20:57:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11F2628498 for ; Tue, 25 Jun 2019 20:57:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726385AbfFYU5l (ORCPT ); Tue, 25 Jun 2019 16:57:41 -0400 Received: from mail-eopbgr140054.outbound.protection.outlook.com ([40.107.14.54]:20873 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726376AbfFYU5k (ORCPT ); Tue, 25 Jun 2019 16:57:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=testarcselector01; d=microsoft.com; cv=none; b=P4bDK9fOQzzZUVClpZtMk91AoYtXXlGm2ujmUvs7gwhnokXY8u4Ue0DRPewUH0yK3tcA2dDqP4GtyLi4iBmK2uJpqRa3WKibfO7bQy9WckM+sJX3lwTdZdPPZYWpYYrGiA6R5EeirWx+TFdhssRdXjL4jNy5DIQDNRgBcU6kWts= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=testarcselector01; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FYPg42gtesq0bzoKy8Q694OjJSFnWggRTBx/aw2TKsA=; b=o92T5gqtPH+WXtxYyzH/cs+eaEIPxMe9HO5PidR9qh7uxCjoqXyLmD+yfKrvUF8XjjvQaeupxKVG/WeiBggtwaSOZAj8xcfRqCtFavAaswJqPgNrBRlQcMyqGhhdU3NM+T8RkazoDx9+OnVfevzVJL3xDViz23kY2GBxe+6S0ls= ARC-Authentication-Results: i=1; test.office365.com 1;spf=none;dmarc=none;dkim=none;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FYPg42gtesq0bzoKy8Q694OjJSFnWggRTBx/aw2TKsA=; b=J775P0Va9z/z6YE8PA8rHJKKVT0wWEtT4SGm0ijTPFwh/iwaiMXKkAB6NgHa4xRqwcdU/QrrUb+PFJJWZ8W1bwiXa83K3KjxprN9uCNWJ9sgak3bqF2R+jRssc/2H1BW6Pu+Gz3v2lxz3g188XRXZskPliml/dgS5XckOMMKwu8= Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com (10.172.227.7) by DB6PR0501MB2696.eurprd05.prod.outlook.com (10.172.225.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.16; Tue, 25 Jun 2019 20:57:30 +0000 Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278]) by DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278%2]) with mapi id 15.20.2008.014; Tue, 25 Jun 2019 20:57:30 +0000 From: Saeed Mahameed To: "David S. Miller" , Doug Ledford , Jason Gunthorpe CC: Leon Romanovsky , Or Gerlitz , Sagi Grimberg , Tal Gilboa , "netdev@vger.kernel.org" , "linux-rdma@vger.kernel.org" , Saeed Mahameed Subject: [for-next V2 01/10] linux/dim: Move logic to dim.h Thread-Topic: [for-next V2 01/10] linux/dim: Move logic to dim.h Thread-Index: AQHVK5iaVhSUhuGko0CwjtWHa5fzZw== Date: Tue, 25 Jun 2019 20:57:30 +0000 Message-ID: <20190625205701.17849-2-saeedm@mellanox.com> References: <20190625205701.17849-1-saeedm@mellanox.com> In-Reply-To: <20190625205701.17849-1-saeedm@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-originating-ip: [209.116.155.178] x-clientproxiedby: BYAPR03CA0025.namprd03.prod.outlook.com (2603:10b6:a02:a8::38) To DB6PR0501MB2759.eurprd05.prod.outlook.com (2603:10a6:4:84::7) authentication-results: spf=none (sender IP is ) smtp.mailfrom=saeedm@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 65c80d18-2896-4921-e7c5-08d6f9afbc9b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:DB6PR0501MB2696; x-ms-traffictypediagnostic: DB6PR0501MB2696: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:262; x-forefront-prvs: 0079056367 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(39860400002)(396003)(136003)(376002)(366004)(346002)(199004)(189003)(6512007)(66476007)(186003)(86362001)(305945005)(3846002)(26005)(6116002)(68736007)(7736002)(102836004)(50226002)(99286004)(53936002)(6636002)(76176011)(1076003)(2906002)(5660300002)(6436002)(64756008)(14454004)(66556008)(25786009)(486006)(52116002)(73956011)(71190400001)(6486002)(6506007)(11346002)(81166006)(71200400001)(446003)(386003)(66946007)(256004)(81156014)(66446008)(478600001)(4326008)(66066001)(8676002)(107886003)(2616005)(36756003)(316002)(476003)(8936002)(110136005)(54906003);DIR:OUT;SFP:1101;SCL:1;SRVR:DB6PR0501MB2696;H:DB6PR0501MB2759.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: nLruHNCglZ235aFC1EUQWSHkMgSjwqy92xREy6fNAzXvdZ4RR6r6ISycz3ZXXRHwD28h5uet02f+70UFLykYTzS1BULiUmS157+ptFqQ2Gn/ot36Y36Op33kpW0+YLppWbyvAGPX3CrTof6zp5oRtTzwx9hHRtyaXAlzOxi1M4fQ43dQvZMQODAk+he+IXlzY9aUc5ouK6V4JUiN4t5g+CQvgsblAw9c0I5SbyhM1Z8hbW0cosNmM/UnmPEKeUhlDSrjTU1G3OrHKlHD7Hp9kqlHdq61yynFapbH43XF52Pu6y1DGV06pdnnMn2BHg8nfxm3Wi6dO7QqvWntzOvm+i121eQ9IJFrlVlcpao65nxlLdDO2uz3HNdJEA8/dP42MqHWLAX0uXey0q9f3OHAELzZghjt4sa1qHEEKtQXY4M= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65c80d18-2896-4921-e7c5-08d6f9afbc9b X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jun 2019 20:57:30.4636 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: saeedm@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2696 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tal Gilboa In preparation for supporting more implementations of the DIM algorithm, I'm moving what would become common logic to a common library. Downstream DIM implementations will use the common lib for their implementation. Signed-off-by: Tal Gilboa Signed-off-by: Saeed Mahameed Reviewed-by: Sagi Grimberg --- MAINTAINERS | 1 + include/linux/dim.h | 153 ++++++++++++++++++++++++++++++++++++++++ include/linux/net_dim.h | 148 +------------------------------------- 3 files changed, 156 insertions(+), 146 deletions(-) create mode 100644 include/linux/dim.h diff --git a/MAINTAINERS b/MAINTAINERS index 429c6c624861..5d4b852d9d39 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5589,6 +5589,7 @@ DYNAMIC INTERRUPT MODERATION M: Tal Gilboa S: Maintained F: include/linux/net_dim.h +F: include/linux/dim.h DZ DECSTATION DZ11 SERIAL DRIVER M: "Maciej W. Rozycki" diff --git a/include/linux/dim.h b/include/linux/dim.h new file mode 100644 index 000000000000..67d7ca40f3dd --- /dev/null +++ b/include/linux/dim.h @@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2019 Mellanox Technologies. */ + +#ifndef DIM_H +#define DIM_H + +#include + +#define NET_DIM_NEVENTS 64 + +/* more than 10% difference */ +#define IS_SIGNIFICANT_DIFF(val, ref) \ + (((100UL * abs((val) - (ref))) / (ref)) > 10) +#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) \ +& (BIT_ULL(bits) - 1)) + +struct net_dim_cq_moder { + u16 usec; + u16 pkts; + u8 cq_period_mode; +}; + +struct net_dim_sample { + ktime_t time; + u32 pkt_ctr; + u32 byte_ctr; + u16 event_ctr; +}; + +struct net_dim_stats { + int ppms; /* packets per msec */ + int bpms; /* bytes per msec */ + int epms; /* events per msec */ +}; + +struct net_dim { /* Dynamic Interrupt Moderation */ + u8 state; + struct net_dim_stats prev_stats; + struct net_dim_sample start_sample; + struct work_struct work; + u8 profile_ix; + u8 mode; + u8 tune_state; + u8 steps_right; + u8 steps_left; + u8 tired; +}; + +enum { + NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0, + NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1, + NET_DIM_CQ_PERIOD_NUM_MODES +}; + +enum { + NET_DIM_START_MEASURE, + NET_DIM_MEASURE_IN_PROGRESS, + NET_DIM_APPLY_NEW_PROFILE, +}; + +enum { + NET_DIM_PARKING_ON_TOP, + NET_DIM_PARKING_TIRED, + NET_DIM_GOING_RIGHT, + NET_DIM_GOING_LEFT, +}; + +enum { + NET_DIM_STATS_WORSE, + NET_DIM_STATS_SAME, + NET_DIM_STATS_BETTER, +}; + +enum { + NET_DIM_STEPPED, + NET_DIM_TOO_TIRED, + NET_DIM_ON_EDGE, +}; + +static inline bool net_dim_on_top(struct net_dim *net_dim) +{ + switch (net_dim->tune_state) { + case NET_DIM_PARKING_ON_TOP: + case NET_DIM_PARKING_TIRED: + return true; + case NET_DIM_GOING_RIGHT: + return (net_dim->steps_left > 1) && (net_dim->steps_right == 1); + default: /* NET_DIM_GOING_LEFT */ + return (net_dim->steps_right > 1) && (net_dim->steps_left == 1); + } +} + +static inline void net_dim_turn(struct net_dim *net_dim) +{ + switch (net_dim->tune_state) { + case NET_DIM_PARKING_ON_TOP: + case NET_DIM_PARKING_TIRED: + break; + case NET_DIM_GOING_RIGHT: + net_dim->tune_state = NET_DIM_GOING_LEFT; + net_dim->steps_left = 0; + break; + case NET_DIM_GOING_LEFT: + net_dim->tune_state = NET_DIM_GOING_RIGHT; + net_dim->steps_right = 0; + break; + } +} + +static inline void net_dim_park_on_top(struct net_dim *net_dim) +{ + net_dim->steps_right = 0; + net_dim->steps_left = 0; + net_dim->tired = 0; + net_dim->tune_state = NET_DIM_PARKING_ON_TOP; +} + +static inline void net_dim_park_tired(struct net_dim *net_dim) +{ + net_dim->steps_right = 0; + net_dim->steps_left = 0; + net_dim->tune_state = NET_DIM_PARKING_TIRED; +} + +static inline void +net_dim_sample(u16 event_ctr, u64 packets, u64 bytes, struct net_dim_sample *s) +{ + s->time = ktime_get(); + s->pkt_ctr = packets; + s->byte_ctr = bytes; + s->event_ctr = event_ctr; +} + +static inline void +net_dim_calc_stats(struct net_dim_sample *start, struct net_dim_sample *end, + struct net_dim_stats *curr_stats) +{ + /* u32 holds up to 71 minutes, should be enough */ + u32 delta_us = ktime_us_delta(end->time, start->time); + u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr); + u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr, + start->byte_ctr); + + if (!delta_us) + return; + + curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us); + curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us); + curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC, + delta_us); +} + +#endif /* DIM_H */ diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h index fd458389f7d1..373cda74b167 100644 --- a/include/linux/net_dim.h +++ b/include/linux/net_dim.h @@ -35,73 +35,10 @@ #define NET_DIM_H #include - -struct net_dim_cq_moder { - u16 usec; - u16 pkts; - u8 cq_period_mode; -}; - -struct net_dim_sample { - ktime_t time; - u32 pkt_ctr; - u32 byte_ctr; - u16 event_ctr; -}; - -struct net_dim_stats { - int ppms; /* packets per msec */ - int bpms; /* bytes per msec */ - int epms; /* events per msec */ -}; - -struct net_dim { /* Adaptive Moderation */ - u8 state; - struct net_dim_stats prev_stats; - struct net_dim_sample start_sample; - struct work_struct work; - u8 profile_ix; - u8 mode; - u8 tune_state; - u8 steps_right; - u8 steps_left; - u8 tired; -}; - -enum { - NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0, - NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1, - NET_DIM_CQ_PERIOD_NUM_MODES -}; - -/* Adaptive moderation logic */ -enum { - NET_DIM_START_MEASURE, - NET_DIM_MEASURE_IN_PROGRESS, - NET_DIM_APPLY_NEW_PROFILE, -}; - -enum { - NET_DIM_PARKING_ON_TOP, - NET_DIM_PARKING_TIRED, - NET_DIM_GOING_RIGHT, - NET_DIM_GOING_LEFT, -}; - -enum { - NET_DIM_STATS_WORSE, - NET_DIM_STATS_SAME, - NET_DIM_STATS_BETTER, -}; - -enum { - NET_DIM_STEPPED, - NET_DIM_TOO_TIRED, - NET_DIM_ON_EDGE, -}; +#include #define NET_DIM_PARAMS_NUM_PROFILES 5 -/* Adaptive moderation profiles */ +/* Netdev dynamic interrupt moderation profiles */ #define NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE 256 #define NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE 128 #define NET_DIM_DEF_PROFILE_CQE 1 @@ -188,36 +125,6 @@ net_dim_get_def_tx_moderation(u8 cq_period_mode) return net_dim_get_tx_moderation(cq_period_mode, profile_ix); } -static inline bool net_dim_on_top(struct net_dim *dim) -{ - switch (dim->tune_state) { - case NET_DIM_PARKING_ON_TOP: - case NET_DIM_PARKING_TIRED: - return true; - case NET_DIM_GOING_RIGHT: - return (dim->steps_left > 1) && (dim->steps_right == 1); - default: /* NET_DIM_GOING_LEFT */ - return (dim->steps_right > 1) && (dim->steps_left == 1); - } -} - -static inline void net_dim_turn(struct net_dim *dim) -{ - switch (dim->tune_state) { - case NET_DIM_PARKING_ON_TOP: - case NET_DIM_PARKING_TIRED: - break; - case NET_DIM_GOING_RIGHT: - dim->tune_state = NET_DIM_GOING_LEFT; - dim->steps_left = 0; - break; - case NET_DIM_GOING_LEFT: - dim->tune_state = NET_DIM_GOING_RIGHT; - dim->steps_right = 0; - break; - } -} - static inline int net_dim_step(struct net_dim *dim) { if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2)) @@ -245,21 +152,6 @@ static inline int net_dim_step(struct net_dim *dim) return NET_DIM_STEPPED; } -static inline void net_dim_park_on_top(struct net_dim *dim) -{ - dim->steps_right = 0; - dim->steps_left = 0; - dim->tired = 0; - dim->tune_state = NET_DIM_PARKING_ON_TOP; -} - -static inline void net_dim_park_tired(struct net_dim *dim) -{ - dim->steps_right = 0; - dim->steps_left = 0; - dim->tune_state = NET_DIM_PARKING_TIRED; -} - static inline void net_dim_exit_parking(struct net_dim *dim) { dim->tune_state = dim->profile_ix ? NET_DIM_GOING_LEFT : @@ -267,9 +159,6 @@ static inline void net_dim_exit_parking(struct net_dim *dim) net_dim_step(dim); } -#define IS_SIGNIFICANT_DIFF(val, ref) \ - (((100UL * abs((val) - (ref))) / (ref)) > 10) /* more than 10% difference */ - static inline int net_dim_stats_compare(struct net_dim_stats *curr, struct net_dim_stats *prev) { @@ -351,39 +240,6 @@ static inline bool net_dim_decision(struct net_dim_stats *curr_stats, return dim->profile_ix != prev_ix; } -static inline void net_dim_sample(u16 event_ctr, - u64 packets, - u64 bytes, - struct net_dim_sample *s) -{ - s->time = ktime_get(); - s->pkt_ctr = packets; - s->byte_ctr = bytes; - s->event_ctr = event_ctr; -} - -#define NET_DIM_NEVENTS 64 -#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1)) - -static inline void net_dim_calc_stats(struct net_dim_sample *start, - struct net_dim_sample *end, - struct net_dim_stats *curr_stats) -{ - /* u32 holds up to 71 minutes, should be enough */ - u32 delta_us = ktime_us_delta(end->time, start->time); - u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr); - u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr, - start->byte_ctr); - - if (!delta_us) - return; - - curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us); - curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us); - curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC, - delta_us); -} - static inline void net_dim(struct net_dim *dim, struct net_dim_sample end_sample) {