From patchwork Wed Feb 19 17:22:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982564 Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EC2F2144BE for ; Wed, 19 Feb 2025 17:26:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985967; cv=none; b=fmT4JYMlvTYkRXHduNqHVbPOet3btezNeXZ8p9TqGWQicwy4Jgj7H7hS6qlnFR7StgtoQ0BvSZcyH2eXNRNJpORCnRpXgzPhNdnI2ZSBcYqTGTKOozR2GXdZsakqwBLMY09iLmLa8JXzty22xCXrX3OCMjA06QO/wy7vKE+l9s8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985967; c=relaxed/simple; bh=ncIk5lZdtXMareZLLRLznXToBtthtk/Cr53z96DFIhE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rMxefQAw52xiv//T7lqkloouH/J1b1EvMBseUf6KlFRtecL7K+y8ERr7ncalKF9CbsShL+rC8EiJthub6VFgsGaxuylQy3Gc7GWS1kIDU8DnyX1VKMlqLCVbwg6wYT/hIXBRe6vPqcbFiFT9LjO6r6uC4nI4xTRQbr3TC+5W5yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=hpTWkeuh; arc=none smtp.client-ip=209.85.166.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="hpTWkeuh" Received: by mail-io1-f47.google.com with SMTP id ca18e2360f4ac-855b3e42c5bso790139f.3 for ; Wed, 19 Feb 2025 09:26:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985965; x=1740590765; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B0lcG2A37eoJmBm5Sthc9LwQizBsGAmnbEYmyOv84yU=; b=hpTWkeuhigZzu26F/jVaGSUM4d8OQOkOogJ1TW3UtqtNsN5h9GrW3HIBGqwkA49MEV U6/NWIxedO1TCL5gnPYP6ah/2K+Limag27Nq09AoTf0eTkJFKMR0MggZfTUQdYkBNcNM sGSHoHyJQ49g2o5keRuiYFkVaMrzyRPsmogkAVse/Y53duT8V/cTnYuxQF1HPLP8Xwqg CCnrNPMmv0GAVCIJjooqXOB8mbIOU72twwzqifKiBIkwQNCd7aRYSfTZm8zSJ/aOTcWR ZowSQ29Np4r4rfIE3ytdt9DiYBbLw/hpfCTuRrEiM5MHNmxhe5Fb+VtrG8J6qNYKhCeO 5CLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985965; x=1740590765; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B0lcG2A37eoJmBm5Sthc9LwQizBsGAmnbEYmyOv84yU=; b=MS8TEJ1qR7f+7WIDtzk4daHeh4X5StVQnTf8oOGKqUIf+VnpDBVyMQCOboIAIUeQhi hhnE6Keq+JTtU0mAuNxj16TG2hjx3XIaJQxae5VaYBVlYA8WBLbt23400EI54ixeocxn 4EoLH0hm0g7EbqJVK5y91nP5NtcQfXEYU0CoYjL6rOgUAkc3pKVfKRpwzaq3ouA5uqCE C+wywasU1lJ68jscf76fHd/STiVpizxjImihy/GwtrKdDiunca+VzLFjH6CvjbX4EgiE VbYmGb7Lm2YvKpvQ+0Y25FdyRWXS/FmvejY2C6PeRTLX4MH+/bGePvH6VPYxFYxwxzmr tudQ== X-Gm-Message-State: AOJu0YyHpOL79XCFlIMB2Vv0Lte0XZAltp7+dAXcOFSm6t9fTzjXX4Ty 4EHmKyLg8+9YX11um9/Chm5jfvbvfOsnB3q8eOFRpoAhX90Ooab0An6rieRV4VLp6KskgQOlDTs o X-Gm-Gg: ASbGnctvq+2gasYN8kYNN9NxfGmaYZTXOVbSWSvpHQBsMyNmvhoJr8NK613hcWnQ9sL ERdn7wJtM9dVgT2LgFlicz9bvguViUfM5ZRJVrDqRVSrW3FvzpvHMUyRwCfFJHL9ZdvR/4XFuE/ CiCgZmpJU0lszoBUl48EmcgOBBOPHhcjlcy5XBhVK1V+IoA3rKyUQ34LFHEMqgHM78OdR9m/bsI eg9etReqYuTZq2AAWZ8e1wslIwfK+pK3MrEMXMAfRbB4qDO+WM3TnTNBX2ctsO/zB2Sjkbv42Dj 866NeDfroWO41S/x38M= X-Google-Smtp-Source: AGHT+IGYIPWi9eQLHYtKf1drbY2zK8Yd0TsYlqHyvQz2uzX5viGmpOXBKHoGJnimabLO1Cg8mKr3uA== X-Received: by 2002:a05:6602:1346:b0:855:b8c0:8639 with SMTP id ca18e2360f4ac-855b8c08f1dmr326646939f.14.1739985965016; Wed, 19 Feb 2025 09:26:05 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:04 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 1/5] eventpoll: abstract out parameter sanity checking Date: Wed, 19 Feb 2025 10:22:24 -0700 Message-ID: <20250219172552.1565603-2-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a helper that checks the validity of the file descriptor and other parameters passed in to epoll_wait(). Signed-off-by: Jens Axboe --- fs/eventpoll.c | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 7c0980db77b3..565bf451df82 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2445,6 +2445,27 @@ SYSCALL_DEFINE4(epoll_ctl, int, epfd, int, op, int, fd, return do_epoll_ctl(epfd, op, fd, &epds, false); } +static int ep_check_params(struct file *file, struct epoll_event __user *evs, + int maxevents) +{ + /* The maximum number of event must be greater than zero */ + if (maxevents <= 0 || maxevents > EP_MAX_EVENTS) + return -EINVAL; + + /* Verify that the area passed by the user is writeable */ + if (!access_ok(evs, maxevents * sizeof(struct epoll_event))) + return -EFAULT; + + /* + * We have to check that the file structure underneath the fd + * the user passed to us _is_ an eventpoll file. + */ + if (!is_file_epoll(file)) + return -EINVAL; + + return 0; +} + /* * Implement the event wait interface for the eventpoll file. It is the kernel * part of the user space epoll_wait(2). @@ -2453,26 +2474,16 @@ static int do_epoll_wait(int epfd, struct epoll_event __user *events, int maxevents, struct timespec64 *to) { struct eventpoll *ep; - - /* The maximum number of event must be greater than zero */ - if (maxevents <= 0 || maxevents > EP_MAX_EVENTS) - return -EINVAL; - - /* Verify that the area passed by the user is writeable */ - if (!access_ok(events, maxevents * sizeof(struct epoll_event))) - return -EFAULT; + int ret; /* Get the "struct file *" for the eventpoll file */ CLASS(fd, f)(epfd); if (fd_empty(f)) return -EBADF; - /* - * We have to check that the file structure underneath the fd - * the user passed to us _is_ an eventpoll file. - */ - if (!is_file_epoll(fd_file(f))) - return -EINVAL; + ret = ep_check_params(fd_file(f), events, maxevents); + if (unlikely(ret)) + return ret; /* * At this point it is safe to assume that the "private_data" contains From patchwork Wed Feb 19 17:22:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982565 Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BA352144DC for ; Wed, 19 Feb 2025 17:26:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985968; cv=none; b=ivktAayseSFmibxIhuArrQS4IIHHlxH9k1D2vrjlP11ojpPnuCd+PsBl7kCj/bsuyZWhLet5a5zAWkPdkPpY7oCXi84VmAxcXe91U7tlyGepLbJJ7Xhlc3M9tRqyxEc4c+N2frTCLTr/nawdx6IP06qh5dFNSkNGRpgahQXGI1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985968; c=relaxed/simple; bh=t9VFdL/JGSkI1kK+B8jwU1v2EPjZm47f8WEHapLZ7wc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ghF2o/9Imq7rmmnRpYsn3d4a8uDX4/6xmnxSLW5RbYODuYdDeviu/uOtAf/YciUyOix6jhmR/1s2Z09OJr118ICfOJo72DTGIUcwnXT2vqn2MJgQGzMwG68qOKiNx4piN+syVLmCURvIFxX7Z5yGK8lAvCjXeEibRHyVKesMZ9s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=enej1hDr; arc=none smtp.client-ip=209.85.166.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="enej1hDr" Received: by mail-io1-f53.google.com with SMTP id ca18e2360f4ac-855b3e42c5bso791039f.3 for ; Wed, 19 Feb 2025 09:26:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985966; x=1740590766; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1wj/NWWDbHBPUQsB02XJgoAK7FnxuSmVDgGIKejleZ4=; b=enej1hDrhmuIYdeMJy9bUwnuXUsVtUYDEenZZEF+QcABBK6YPAeQdNg+j7Liv4VfJX UwCTNtGWykuq1SpvuONFSwiddML7oRkhNELvDT3KM0WQ7DcYvejm7SHGjzw06OCs/pOF uXpNxrWzEd2lEVf+hM1HaRqnvtGe+F97FIWaU2G073LBJJSHgwgoI6FB6xHQhMddLb77 WevOOj51aY2dT3S9GSVWQGJX7LT3d9DrkPgRVMRsviqWEixl3RcIF2jQ5dgxa6J3HWK9 eLLK+MredZWWLudxj+bf7HVrS9tlBQmwbBmsfr++rLlCIVLW3AXfKP7u1JIUwsB3RUO2 3g+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985966; x=1740590766; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1wj/NWWDbHBPUQsB02XJgoAK7FnxuSmVDgGIKejleZ4=; b=dfvoBjOgZv1QNLJe8TbHl2jFc0E6opHz4nwqRRzMYOupdV8HP3Oxvp8LI/3ZszOuGK Ef9uCtpgzGS/au4EspX3UKI0oPHuw9E2QgsSJzARJCHV9xD/xhfnRnpe36X1wRGXB0Vg pqP4VbRPoaCxlikVXohUIaTS6RdevZubeLT1V8cBp93nHAkp12KKvIb4f9PghZEsyUn+ YT6T6Filfu5TMUOFH8LKLNGXbkaIJxBYKquyUKQUYmF7spOMGo75T4R7mz5EX9faLQVS coo8tlnty+N0S3A/GO7Wr8itwAgxs/FRGZZhvVh8NyZ9j/x2RZ43VKxJWc47R4hHWr4C oIRQ== X-Gm-Message-State: AOJu0YxVaQIFfxWshTe/95AAO8YYluxUE+mJ253r2ja3dsoa4vHC7Wxu RIIzrGHRI0fIcaTr1yNwLlKpgzVwK/zuJ2UPTX23zA9YWaVD/MzYfcwGsdwL4wU7Rt7TYcDncWg t X-Gm-Gg: ASbGnctKUFQXlCR1HMvAGNgxeAqg/z9HKS6CxOgRyerDuwwzKrgTO2LGltVZnKvIHDX nV1buB1FvxwMCd+oL86se6MlpP9IvYAfds2jiuvxF4FifcCKw8jv5ul1izW4Vh4OXNMJPURgxFu p7yXBpiVYUNJs1ym179vZ0UbYetsbFnOThMfXHh5o7ilJ8nC2pCYCXcKTFu43MuJVIfEnMIsbzS 2Kzikowe3o8G5PSUOMqTkYozP1icWnzBBWG9vwuH9scAdIHm2e7akMKMzUkjNNso06q0vugEq0d 506iQkmTYA8Y+nmdX9g= X-Google-Smtp-Source: AGHT+IFtaL9/XuB8qrh/MkYo4ESV+gFWdA1YcEjvsmOMak+8t8dipfJF4xY48fkZWILi3KWxQmI7cA== X-Received: by 2002:a05:6602:6d8d:b0:855:b5fe:3fb7 with SMTP id ca18e2360f4ac-855b5fe4041mr283067439f.7.1739985966274; Wed, 19 Feb 2025 09:26:06 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:05 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 2/5] eventpoll: abstract out ep_try_send_events() helper Date: Wed, 19 Feb 2025 10:22:25 -0700 Message-ID: <20250219172552.1565603-3-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation for reusing this helper in another epoll setup helper, abstract it out. Signed-off-by: Jens Axboe --- fs/eventpoll.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 565bf451df82..14466765b85d 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -1980,6 +1980,22 @@ static int ep_autoremove_wake_function(struct wait_queue_entry *wq_entry, return ret; } +static int ep_try_send_events(struct eventpoll *ep, + struct epoll_event __user *events, int maxevents) +{ + int res; + + /* + * Try to transfer events to user space. In case we get 0 events and + * there's still timeout left over, we go trying again in search of + * more luck. + */ + res = ep_send_events(ep, events, maxevents); + if (res > 0) + ep_suspend_napi_irqs(ep); + return res; +} + /** * ep_poll - Retrieves ready events, and delivers them to the caller-supplied * event buffer. @@ -2031,17 +2047,9 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, while (1) { if (eavail) { - /* - * Try to transfer events to user space. In case we get - * 0 events and there's still timeout left over, we go - * trying again in search of more luck. - */ - res = ep_send_events(ep, events, maxevents); - if (res) { - if (res > 0) - ep_suspend_napi_irqs(ep); + res = ep_try_send_events(ep, events, maxevents); + if (res) return res; - } } if (timed_out) From patchwork Wed Feb 19 17:22:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982566 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D16B21423F for ; Wed, 19 Feb 2025 17:26:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985970; cv=none; b=ZaOhg3bsecG2j1ZzDvw5fY73Uh4u1/iMS40HDVaxVlBQziLnH044eEdtdXJmp2txGdmvHGPfnTrlfVK18zrCm+2P/p+v1nAncpsfW9XGCMxbvIQuvVzHhc7YlnnXrNJH4yrqQO7py+1k5i9/8Rl2Y4hfdbf8k+HdDqRudN6ieXY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985970; c=relaxed/simple; bh=N/K1wXHto9EUav/plQ5H1Ct3sazV7rp7nIQXrrPuZM0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pFEtpN2fCiFH/7Zn30oaydtn1cfUSKa4yt2SzX5tBsv7YcmnKYHNfR/2kCZeSIUuE6aSTeCmhd/ApJIpTzmjGzPrk4biaflIEms9slsR0QPz2vkxd7vu8zjmP253e8PmL0eZTM9aJGerTxdheGoKrohXxUBydq4jA4cdl+ArNp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=mTgOMfe9; arc=none smtp.client-ip=209.85.166.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="mTgOMfe9" Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-85598a3e64bso3007239f.1 for ; Wed, 19 Feb 2025 09:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985967; x=1740590767; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3hLS+fhEeJItsy0qr9n/8T/ts3J225prk6HY4IC/ZoU=; b=mTgOMfe9wwXFATa4JuF9Fbk7zmVjBpQzVsvBlES10VGQAlzPC7PoNgM/egWxRo6WZW EOKGho6Inbo4gIRdRskyfHIVBOg+4PhyiKRQSQjKdSWEhjsH+Uq03XsMDac7U22VvZVe w1M+JF7IK3/DXHSXBjopLOv3aP3o2np3Qsoq7zkMHs9bKwv/uSUczFWa5sru7pTAgS9Y bUy0eXXxy+r6ZsN4KuBHpKcRvzBJ5Fwi5UbcLHtH4nm8DjsfIJfzFIFa0K3ku0LJkLUu YInwP9Wei2royHhI0knm92Th6pF1WzotToboXK4h8X369OlUm2DM1vw/Yy9/3+dc8ldW I6jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985967; x=1740590767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3hLS+fhEeJItsy0qr9n/8T/ts3J225prk6HY4IC/ZoU=; b=V0VDDoXkcJx5JnWUDOTCoVfYoc+GoxTyWmBrRVu1Sy+nAxYW/wbADbXSzSEHFALrnu 0G7/E2NRwB3xVEJX8j8H4LkrJqvhKqgzvP1RPfy7zYITZWhBmfqEj7N8KZZukJNNvs61 SxfqeWaqzcd6z7PiOuM8IxkYbDpJa14H9nHCb5Ev/2TZe05xZAxzrjwNv69P2lno14xN jG8170u7/A+jChiMjI4VfjthkzYIDgt+0bnBdjKZPp37RYPxKTDuwgDNrCtmj2485Nyd 0HCHKJAPrsZQgmse487bP9LxQ47IcQxEQwEWfveiY41tFKTS2H9pSojBuzIpFwcAm9lU ohPg== X-Gm-Message-State: AOJu0YyxhbVZSjB5kA7B/SsAHPoRns0+BwKrZimihW9kdEWzl/5l8KjX llOQOMjW17xUtKJCWPrKcb80XCHjNBy//BJ6GHaJcgDGPLEO/MiclATpH/Eov50= X-Gm-Gg: ASbGncvaF5f/M4SPX2VpbCLhN8knzGgxdJnPLg4p2JcwFY8LrDENmdmGdlMZKcYsmdv 9+E7v08Q/HuHbtM/C1nhjlJ6nm5tO+yrTwQ+qifXApxpZYFtD4NgaHmyqQvND/uebyQKdm14w+c dI2QJ1hCq+CX75q7sPNkXZKSvKFICkBtnF4AiFtAFN1bjpPo8E9FrR5ZHI7Qq/sZHJC3T9zQBrX O/AAiwWWpw1Rm6NXVBAibEExb2dO18WpmuhH0T1jgayQX84lT4RlOPFvs94K9++XRiG3x5QpEWr nwPCjrp2oFBEkO4Yyds= X-Google-Smtp-Source: AGHT+IFu6pkBHN/jzNR4Yx3waXcQL0mHGqKfBL7pK3ptWaTbpFR05iOXURlc6k5WEDumbHgigo/mcQ== X-Received: by 2002:a05:6602:150f:b0:855:a4a4:a938 with SMTP id ca18e2360f4ac-855a4a4a98bmr863821939f.2.1739985967617; Wed, 19 Feb 2025 09:26:07 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:06 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 3/5] eventpoll: add epoll_sendevents() helper Date: Wed, 19 Feb 2025 10:22:26 -0700 Message-ID: <20250219172552.1565603-4-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Basic helper that copies ready events to the specified userspace address. The event checking is quick and racy, it's up to the caller to ensure it retries appropriately in case 0 events are copied. Signed-off-by: Jens Axboe --- fs/eventpoll.c | 20 ++++++++++++++++++++ include/linux/eventpoll.h | 4 ++++ 2 files changed, 24 insertions(+) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 14466765b85d..94b87aaad0f6 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2474,6 +2474,26 @@ static int ep_check_params(struct file *file, struct epoll_event __user *evs, return 0; } +int epoll_sendevents(struct file *file, struct epoll_event __user *events, + int maxevents) +{ + struct eventpoll *ep; + int ret; + + ret = ep_check_params(file, events, maxevents); + if (unlikely(ret)) + return ret; + + ep = file->private_data; + /* + * Racy call, but that's ok - it should get retried based on + * poll readiness anyway. + */ + if (ep_events_available(ep)) + return ep_try_send_events(ep, events, maxevents); + return 0; +} + /* * Implement the event wait interface for the eventpoll file. It is the kernel * part of the user space epoll_wait(2). diff --git a/include/linux/eventpoll.h b/include/linux/eventpoll.h index 0c0d00fcd131..ccb478eb174b 100644 --- a/include/linux/eventpoll.h +++ b/include/linux/eventpoll.h @@ -25,6 +25,10 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, unsigned long t /* Used to release the epoll bits inside the "struct file" */ void eventpoll_release_file(struct file *file); +/* Copy ready events to userspace */ +int epoll_sendevents(struct file *file, struct epoll_event __user *events, + int maxevents); + /* * This is called from inside fs/file_table.c:__fput() to unlink files * from the eventpoll interface. We need to have this facility to cleanup From patchwork Wed Feb 19 17:22:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982567 Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABC962147F1 for ; Wed, 19 Feb 2025 17:26:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985971; cv=none; b=c5wSZFGEbVY/u8WaC2tqF9FaXOHPw5q1Lq8XMCZuBRNaE/tMI3AG4mHFUv/fu633XayMDyBlIEVQ7Z8Z18VUXKXJVvTq8QRzypHwI3AydVQvD63jd8ekIFh4Va3j73vmdgiGnrD5RUs8l9ZEoHlcvHvZkmRHowxCxmg6mgVM7aU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985971; c=relaxed/simple; bh=NKhQ3CZ0sAQuduklLkJ8jHQS9JJUaLrLvLnUNJYM5Q4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WqKsnEz78VVyrC6dIFDsdCCNiMriIbSSfNajOnqukKFgaA4Vdx7bIjuSx+vfE9ZXUAe/JkVNJJ2hajlHeKAOqOQ+/s3h49T4kwVpW8nBuKe1bbPsgGwgFp8rhV+glueEbYEw3si0+XPaMWBkPYvXtCMdvFX3wLfhaq2yt3qoaUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=uv3omP9v; arc=none smtp.client-ip=209.85.166.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="uv3omP9v" Received: by mail-io1-f46.google.com with SMTP id ca18e2360f4ac-852050432a8so1959539f.1 for ; Wed, 19 Feb 2025 09:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985969; x=1740590769; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PqeZNiF4C8ajF9Httgn8ehWaoVM+wUVbzJe1YYgdf+w=; b=uv3omP9vMBs1OH6zr+j8x1E00QLeFQ1bVFZ/3Sr5KU6i2BpRk3+NXpoiTGMEV4e47L TGLPRJqejz24lFFSiqJSNRk8hDz6qSiC8idLdZCwB7bDgKq5Re3jpQCEK8Hc072aUZy6 PgxybNO9fcyOmMJ19KKCJgnmbUWHbuVENCJk2uiQU96ormD9FUpuxIBykblBO3XQT8MM 9/rpTR9GfNKw2+YWYdL9I7M5ss4Knv4hqDblinj7hYPA+LLq5bF3jLvs5zrMdTQLzMkA /5/Si+4dSsFxlyYLIqUWv6OfKL2+w9/VkCmNQTD2p/LnSeNF+0PR+O8buTjSrILX/0FD /ZFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985969; x=1740590769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PqeZNiF4C8ajF9Httgn8ehWaoVM+wUVbzJe1YYgdf+w=; b=hFNQNodJMWQXDHIg7fh9gC3C68hbwoEodiy8Oh5OkkS/5uJGtEeaTrNzzzr12JkYg/ Sq9RlgCwpVvKtgNCU40eD85+Qq3Gpc3lDiZ8oDalr3pJJPxXl/9pJNyFSF5Rxh83BbHV dpgBdkBNIWw98NtTNTgDZhRwGOaSFiAytFLW5kAkWiRPMWgruAqIGe0BUR1h7zKx+LTB SQzz5xYKUxDtRCn4wVqMCNP44pIaTFy3Oe0x93hOfUJTOu9Jk5SmwBveieIWgLTFgPbU wa8Ux4dKnycsIG1t/SnxMHs/p7s7V2vGHEayTEhlWqLYNIqCgheVQ4wXuc++AlmeyBWB 28Fg== X-Gm-Message-State: AOJu0YzjioYxthp+3EvWMTKJ0mIjnE+V/jhPqR706M7n6D8G4gQMrdXb 6YBTqcsMkVF6jkfBIKEi8Dkxyn2iudqVfGJTQbgFR0IGjuCY3+7jhbOiiSyNSW0= X-Gm-Gg: ASbGnctt+YryR69bpCYQxHuGLbArjpblkcqxef0Dg6rd4DYkNsSvbYt2jurutSFzI0p ItGDIOLzrKE3N185kCwi585f+scx29zZAWFCtObLnzqlmcBzWqzSQre5kFI4LNDg+gF6zcb9CVS fiOBtkOpp6AtKdAcZLYyQVxDIdDGhbzb7nAUtTDyrFJxvGafB8fCYYqLwqiZbogK9zXlYVX2JPE yYUOwHTO2hxj5d4jZgaGiczsgB3u+wcqcpGGUISE0zvly+Jk3HkdjzG1geZHgPgroV0JN+is0S3 jeFFkxU/NYy990W91hI= X-Google-Smtp-Source: AGHT+IHRVFWWS5YvlkaNF1Q/MDG4uwtiTei/nIZPfpHN+WkrFtLSjd0dVoryMXxuADhDSJM8pIGxUA== X-Received: by 2002:a05:6602:6d87:b0:855:b0eb:3fe6 with SMTP id ca18e2360f4ac-855c32ce3c0mr10130239f.8.1739985968846; Wed, 19 Feb 2025 09:26:08 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:08 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 4/5] io_uring/epoll: remove CONFIG_EPOLL guards Date: Wed, 19 Feb 2025 10:22:27 -0700 Message-ID: <20250219172552.1565603-5-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Just have the Makefile add the object if epoll is enabled, then it's not necessary to guard the entire epoll.c file inside an CONFIG_EPOLL ifdef. Signed-off-by: Jens Axboe --- io_uring/Makefile | 9 +++++---- io_uring/epoll.c | 2 -- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/io_uring/Makefile b/io_uring/Makefile index 98e48339d84d..3e28a741ca15 100644 --- a/io_uring/Makefile +++ b/io_uring/Makefile @@ -11,10 +11,11 @@ obj-$(CONFIG_IO_URING) += io_uring.o opdef.o kbuf.o rsrc.o notif.o \ eventfd.o uring_cmd.o openclose.o \ sqpoll.o xattr.o nop.o fs.o splice.o \ sync.o msg_ring.o advise.o openclose.o \ - epoll.o statx.o timeout.o fdinfo.o \ - cancel.o waitid.o register.o \ - truncate.o memmap.o alloc_cache.o + statx.o timeout.o fdinfo.o cancel.o \ + waitid.o register.o truncate.o \ + memmap.o alloc_cache.o obj-$(CONFIG_IO_URING_ZCRX) += zcrx.o obj-$(CONFIG_IO_WQ) += io-wq.o obj-$(CONFIG_FUTEX) += futex.o -obj-$(CONFIG_NET_RX_BUSY_POLL) += napi.o +obj-$(CONFIG_EPOLL) += epoll.o +obj-$(CONFIG_NET_RX_BUSY_POLL) += napi.o diff --git a/io_uring/epoll.c b/io_uring/epoll.c index 89bff2068a19..7848d9cc073d 100644 --- a/io_uring/epoll.c +++ b/io_uring/epoll.c @@ -12,7 +12,6 @@ #include "io_uring.h" #include "epoll.h" -#if defined(CONFIG_EPOLL) struct io_epoll { struct file *file; int epfd; @@ -58,4 +57,3 @@ int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags) io_req_set_res(req, ret, 0); return IOU_OK; } -#endif From patchwork Wed Feb 19 17:22:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982568 Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18F8F214A6A for ; Wed, 19 Feb 2025 17:26:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985972; cv=none; b=CkHFT7ieUAl6bFAJZweDKX6NAPm3W8etWCA7MYbWPHF8EtTJnxUzcKIhKdbLcZ/zZflqkr5hrZhju6h5jaX1TTBBzV3GiiwgDlwLREK8aOZL7qQXaImFu3fP3AtYcWjZx1aaUFAKRrCttrnfnkT4lCq4q7n7iNwvQUcLOUNGqmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985972; c=relaxed/simple; bh=Dgu/2bYkWXrAMq3iF1/8GI7X79S6G6gmhyLC1ec2mfY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HS9FkTk5V0dbtpMjzxZknyS98wkFaDwQCI2Qvxg8FiS/cD4rE374tU6X1gCPyq4jw5THhdsI3IVPyzA2sHIl8wslGwyLZsB8hpj70YaQTKQNJWmUt4EDbR0GToiOmSYSutEnKoEtW4ULB2AizjkSkXv9BVhRZCVVX85JJf5AFxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=C2nFRDgx; arc=none smtp.client-ip=209.85.166.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="C2nFRDgx" Received: by mail-io1-f54.google.com with SMTP id ca18e2360f4ac-854a68f5aeeso1110239f.0 for ; Wed, 19 Feb 2025 09:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985970; x=1740590770; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1gAHZg9kSYTvKQ33H52aoGebUTskPe032Qb9P2hQu70=; b=C2nFRDgxkeUJxOFus0XdLUwXUYVZZEsSkdAI1XHXzqRNSSQQmrJD9OXmqK81dXZ+BU niwem1/kEPxvdxUZ4YiskDmNk3XSt7OfAbpvEKkdN7ArTzI6ot1FF25GTJ5yH7Pn9+uZ flvqOxHUsldj7pQwY3oAWiOs9wBRDD647+RID/uADI4JAAEs46G5JA7AsA7MMk5HErtN XhWEQDxMA0mD9Ygz5zU2DfKt8w2L6YiZuftaXcoJiiDHarie7tDMknHTp/GULPB2F199 xFwnXfySPzu1SCSPP34lIziFl53ES2XTG+iMHkehQXmKmPF++FLWcvkUio6s1clC0SOr tiAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985970; x=1740590770; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1gAHZg9kSYTvKQ33H52aoGebUTskPe032Qb9P2hQu70=; b=B4j7Wg89dz0nqDfgwzTK4HNd6MViN361thmAmvvvh3KcS0QbcFLv+PGQqa1LuKMXn1 vTNBWNt+8MU7R1HsrPFjXr+DMZVoRZO9ZZjvfkD7XUZ7h4f0alS2KQoiXRDqwOAU0TsM UDBMgzDaM+c/hxv81krOon4tdbk7AEs4BQSG4PuR70dNrRWq/ZTbqXDbV6ODIWVpkVbr BL33JYMvDrckQrGnmLSD2kbuY2k0OHrdQGKSIdwKzy4RwpK+LG2jkFqt4QOfbsNH+ZlA s+Mh81WgpSzidEmAwBjBjpwbEkMF1luhVfAgnkH5xzctXPSkwLzEItYtuoztg00mMOtw +TBg== X-Gm-Message-State: AOJu0Yx6bUOOf0haMEbOgAd9FpzVLF5fYS2Eluv50SL6FP4MOOKRmAV+ HCqeaJt3g3920dv37ekKPdQv20RXXWB+JeoKi0qdnMIopygB+3rnjQytP+kuYNY= X-Gm-Gg: ASbGncu0FDcmVVBl245lRuKtFdi668TUD1R7pgHG2xCSHs94UbRACqZ0G5vA5iE1gKG qcJOhdI7DjZVF5SHBBj13gHC/cODP6zo5p2jt5bUvdvSCfRUtcTy1ucWmgxxAA60NQ8IDfJatTZ VP8FFpeUEghKTxsvMPuFYAuEEbu+LAozu6HI318b0NJh8sbU+wTzDZxG+aXlExktC2kDJisEJsl ITrgM4sqDEv0KOqgJF/P+blta4y2NsWkp8zpzr5NIiwc36baYw4HBqsIiSoG0slQRIO1nn0+tcB qNPTDiBKEvX4Vx8jShc= X-Google-Smtp-Source: AGHT+IHjnt3iA5lsKLTSE6k9kN1FP0fHcpHTu6e9TZVmjZOfhdBtx2VkZxFxTUM+se6BvwdrScUk8Q== X-Received: by 2002:a05:6602:6d8d:b0:855:b5fe:3fb7 with SMTP id ca18e2360f4ac-855b5fe4041mr283100139f.7.1739985970155; Wed, 19 Feb 2025 09:26:10 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:09 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 5/5] io_uring/epoll: add support for IORING_OP_EPOLL_WAIT Date: Wed, 19 Feb 2025 10:22:28 -0700 Message-ID: <20250219172552.1565603-6-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For existing epoll event loops that can't fully convert to io_uring, the used approach is usually to add the io_uring fd to the epoll instance and use epoll_wait() to wait on both "legacy" and io_uring events. While this work, it isn't optimal as: 1) epoll_wait() is pretty limited in what it can do. It does not support partial reaping of events, or waiting on a batch of events. 2) When an io_uring ring is added to an epoll instance, it activates the io_uring "I'm being polled" logic which slows things down. Rather than use this approach, with EPOLL_WAIT support added to io_uring, event loops can use the normal io_uring wait logic for everything, as long as an epoll wait request has been armed with io_uring. Note that IORING_OP_EPOLL_WAIT does NOT take a timeout value, as this is an async request. Waiting on io_uring events in general has various timeout parameters, and those are the ones that should be used when waiting on any kind of request. If events are immediately available for reaping, then This opcode will return those immediately. If none are available, then it will post an async completion when they become available. cqe->res will contain either an error code (< 0 value) for a malformed request, invalid epoll instance, etc. It will return a positive result indicating how many events were reaped. IORING_OP_EPOLL_WAIT requests may be canceled using the normal io_uring cancelation infrastructure. Signed-off-by: Jens Axboe --- include/uapi/linux/io_uring.h | 1 + io_uring/epoll.c | 33 +++++++++++++++++++++++++++++++++ io_uring/epoll.h | 2 ++ io_uring/opdef.c | 14 ++++++++++++++ 4 files changed, 50 insertions(+) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 05d6255b0f6a..135eb9296296 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -280,6 +280,7 @@ enum io_uring_op { IORING_OP_BIND, IORING_OP_LISTEN, IORING_OP_RECV_ZC, + IORING_OP_EPOLL_WAIT, /* this goes last, obviously */ IORING_OP_LAST, diff --git a/io_uring/epoll.c b/io_uring/epoll.c index 7848d9cc073d..6d2c48ba1923 100644 --- a/io_uring/epoll.c +++ b/io_uring/epoll.c @@ -20,6 +20,12 @@ struct io_epoll { struct epoll_event event; }; +struct io_epoll_wait { + struct file *file; + int maxevents; + struct epoll_event __user *events; +}; + int io_epoll_ctl_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_epoll *epoll = io_kiocb_to_cmd(req, struct io_epoll); @@ -57,3 +63,30 @@ int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags) io_req_set_res(req, ret, 0); return IOU_OK; } + +int io_epoll_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) +{ + struct io_epoll_wait *iew = io_kiocb_to_cmd(req, struct io_epoll_wait); + + if (sqe->off || sqe->rw_flags || sqe->buf_index || sqe->splice_fd_in) + return -EINVAL; + + iew->maxevents = READ_ONCE(sqe->len); + iew->events = u64_to_user_ptr(READ_ONCE(sqe->addr)); + return 0; +} + +int io_epoll_wait(struct io_kiocb *req, unsigned int issue_flags) +{ + struct io_epoll_wait *iew = io_kiocb_to_cmd(req, struct io_epoll_wait); + int ret; + + ret = epoll_sendevents(req->file, iew->events, iew->maxevents); + if (ret == 0) + return -EAGAIN; + if (ret < 0) + req_set_fail(req); + + io_req_set_res(req, ret, 0); + return IOU_OK; +} diff --git a/io_uring/epoll.h b/io_uring/epoll.h index 870cce11ba98..4111997c360b 100644 --- a/io_uring/epoll.h +++ b/io_uring/epoll.h @@ -3,4 +3,6 @@ #if defined(CONFIG_EPOLL) int io_epoll_ctl_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags); +int io_epoll_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); +int io_epoll_wait(struct io_kiocb *req, unsigned int issue_flags); #endif diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 89f50ecadeaf..9344534780a0 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -527,6 +527,17 @@ const struct io_issue_def io_issue_defs[] = { .issue = io_recvzc, #else .prep = io_eopnotsupp_prep, +#endif + }, + [IORING_OP_EPOLL_WAIT] = { + .needs_file = 1, + .audit_skip = 1, + .pollin = 1, +#if defined(CONFIG_EPOLL) + .prep = io_epoll_wait_prep, + .issue = io_epoll_wait, +#else + .prep = io_eopnotsupp_prep, #endif }, }; @@ -761,6 +772,9 @@ const struct io_cold_def io_cold_defs[] = { [IORING_OP_RECV_ZC] = { .name = "RECV_ZC", }, + [IORING_OP_EPOLL_WAIT] = { + .name = "EPOLL_WAIT", + }, }; const char *io_uring_get_opcode(u8 opcode)