From patchwork Wed Feb 19 17:22:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982571 Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FC402144C8 for ; Wed, 19 Feb 2025 17:26:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985967; cv=none; b=ZlYADRWX9BVYQhYYt2iAkUBfyvAGfY69yAUUhKM7w7zZM3mpeJVknC2tMFuuJFD62Kxhj1mc52Ph8Omyue/WmBmPtrmCJUV+x0VC+Fleq5mQ92GhEWX3Y+0cQ1XmWnvGulJimsK4yF7zjillYUPAUkBJhMelMqCfIE6t/bTaSgI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985967; c=relaxed/simple; bh=ncIk5lZdtXMareZLLRLznXToBtthtk/Cr53z96DFIhE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rMxefQAw52xiv//T7lqkloouH/J1b1EvMBseUf6KlFRtecL7K+y8ERr7ncalKF9CbsShL+rC8EiJthub6VFgsGaxuylQy3Gc7GWS1kIDU8DnyX1VKMlqLCVbwg6wYT/hIXBRe6vPqcbFiFT9LjO6r6uC4nI4xTRQbr3TC+5W5yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=hpTWkeuh; arc=none smtp.client-ip=209.85.166.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="hpTWkeuh" Received: by mail-io1-f49.google.com with SMTP id ca18e2360f4ac-8559020a76aso1012339f.2 for ; Wed, 19 Feb 2025 09:26:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985965; x=1740590765; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B0lcG2A37eoJmBm5Sthc9LwQizBsGAmnbEYmyOv84yU=; b=hpTWkeuhigZzu26F/jVaGSUM4d8OQOkOogJ1TW3UtqtNsN5h9GrW3HIBGqwkA49MEV U6/NWIxedO1TCL5gnPYP6ah/2K+Limag27Nq09AoTf0eTkJFKMR0MggZfTUQdYkBNcNM sGSHoHyJQ49g2o5keRuiYFkVaMrzyRPsmogkAVse/Y53duT8V/cTnYuxQF1HPLP8Xwqg CCnrNPMmv0GAVCIJjooqXOB8mbIOU72twwzqifKiBIkwQNCd7aRYSfTZm8zSJ/aOTcWR ZowSQ29Np4r4rfIE3ytdt9DiYBbLw/hpfCTuRrEiM5MHNmxhe5Fb+VtrG8J6qNYKhCeO 5CLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985965; x=1740590765; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B0lcG2A37eoJmBm5Sthc9LwQizBsGAmnbEYmyOv84yU=; b=A9jPfPlP0LK/6JK8aze8EcsimRGRS7+pubzcdFoUmUCLdL9i1RJ0b8kNa4bp9DW/jp e4sOzgdm71jt+83y60hchupQy+1T87sq3zGxQHe3H9Vgqn2Zga/390cXlfF7mdsY/9As iOQMIL9IoysqGiC7wJoSwY1pqqXRbEysPggAAgiSvo0Z7r9zJYyEwtRh+DDmDLhes52v qyPE8PpQZl4vRjF5JE9CEEs0sxCw51UFRtKjM0x0MTo3LMQ0zwwTfWpTEpwmzoStPeD2 jp5i/W6A2YdashIO1ddgJyqzirSuXXOJ7VRc5obXT49uxGkyZpi+NYC9tz6cG2xpvqnA Jr/A== X-Gm-Message-State: AOJu0Yy0RaSqjPRLYbCfbLXW6bfuK/diIvr7+lCT394AGbhgOmNkj8C0 a29a5Wu/1Qe4YkPevvmjowavIhCG3lr5fUejPJQZhP6Q2eFvfzZsOWwXVVTqcKL3HcwHLGHQpwf l X-Gm-Gg: ASbGnctXVsqhSxI3oC9NMwOTEEMaXTD8ACnjz7FpQ+z7x3lEUrbrnwGDAWxNQfu8MvS Yn2AOgc8D9YEB4SfCmcUVhtjtGhJ4Mc9V4bhmjUsydNJ/gdxWur9lpZ/c02CTB0nUh4PQ5NHZ21 PZvsR5Ex7xCbUn01Ns/tYX7sUDXM1WWKP2Ix2yum2zaETt9jWFjriETsdVAE8/q/LNu+0EFHLc5 RQSiZYHRr5fdJB28yQ398Ck+pwQC9uyLI8l4JIM8y/RUM5UTvGD/d0x8F9XjauMB51e1G/F69Na OV4hMFaHpHcRV3TSZzM= X-Google-Smtp-Source: AGHT+IGYIPWi9eQLHYtKf1drbY2zK8Yd0TsYlqHyvQz2uzX5viGmpOXBKHoGJnimabLO1Cg8mKr3uA== X-Received: by 2002:a05:6602:1346:b0:855:b8c0:8639 with SMTP id ca18e2360f4ac-855b8c08f1dmr326646939f.14.1739985965016; Wed, 19 Feb 2025 09:26:05 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:04 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 1/5] eventpoll: abstract out parameter sanity checking Date: Wed, 19 Feb 2025 10:22:24 -0700 Message-ID: <20250219172552.1565603-2-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a helper that checks the validity of the file descriptor and other parameters passed in to epoll_wait(). Signed-off-by: Jens Axboe --- fs/eventpoll.c | 39 +++++++++++++++++++++++++-------------- 1 file changed, 25 insertions(+), 14 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 7c0980db77b3..565bf451df82 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2445,6 +2445,27 @@ SYSCALL_DEFINE4(epoll_ctl, int, epfd, int, op, int, fd, return do_epoll_ctl(epfd, op, fd, &epds, false); } +static int ep_check_params(struct file *file, struct epoll_event __user *evs, + int maxevents) +{ + /* The maximum number of event must be greater than zero */ + if (maxevents <= 0 || maxevents > EP_MAX_EVENTS) + return -EINVAL; + + /* Verify that the area passed by the user is writeable */ + if (!access_ok(evs, maxevents * sizeof(struct epoll_event))) + return -EFAULT; + + /* + * We have to check that the file structure underneath the fd + * the user passed to us _is_ an eventpoll file. + */ + if (!is_file_epoll(file)) + return -EINVAL; + + return 0; +} + /* * Implement the event wait interface for the eventpoll file. It is the kernel * part of the user space epoll_wait(2). @@ -2453,26 +2474,16 @@ static int do_epoll_wait(int epfd, struct epoll_event __user *events, int maxevents, struct timespec64 *to) { struct eventpoll *ep; - - /* The maximum number of event must be greater than zero */ - if (maxevents <= 0 || maxevents > EP_MAX_EVENTS) - return -EINVAL; - - /* Verify that the area passed by the user is writeable */ - if (!access_ok(events, maxevents * sizeof(struct epoll_event))) - return -EFAULT; + int ret; /* Get the "struct file *" for the eventpoll file */ CLASS(fd, f)(epfd); if (fd_empty(f)) return -EBADF; - /* - * We have to check that the file structure underneath the fd - * the user passed to us _is_ an eventpoll file. - */ - if (!is_file_epoll(fd_file(f))) - return -EINVAL; + ret = ep_check_params(fd_file(f), events, maxevents); + if (unlikely(ret)) + return ret; /* * At this point it is safe to assume that the "private_data" contains From patchwork Wed Feb 19 17:22:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982572 Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A098F20E6F9 for ; Wed, 19 Feb 2025 17:26:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985969; cv=none; b=X2qZQTxrlzib7x4x3VeMKyyDusT4Pr9kuNQQgehfEP9cEc8r70v1N8tkPvKLC9F+NytyMv/kjmnbOcWkcRSQhhMTcv5UK5nabDbpvIjLPn9mFTj5Z5SdlGhTwGXfxgyJoGGF+67L+cIJoapOI6smKfQAEiuNyGfEISY/UXn3XGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985969; c=relaxed/simple; bh=t9VFdL/JGSkI1kK+B8jwU1v2EPjZm47f8WEHapLZ7wc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MMbDk7deq9biu1UBC+3ZP9P5xwSPzYIxPYlMDvJNLwaq+lnOQeKlX8su89GYELxEbT4BgMY50MVag/DGZ0v4K5EBZoWpL8KqrgjncRxXW0vZn1vucsidd3KcRwzz2QTp1w7CBJOsS9ZUsny7ztaeiWUSbqJMwRejIKl/7/JcgRE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=enej1hDr; arc=none smtp.client-ip=209.85.166.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="enej1hDr" Received: by mail-io1-f48.google.com with SMTP id ca18e2360f4ac-855a7e3be0eso1084639f.1 for ; Wed, 19 Feb 2025 09:26:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985966; x=1740590766; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1wj/NWWDbHBPUQsB02XJgoAK7FnxuSmVDgGIKejleZ4=; b=enej1hDrhmuIYdeMJy9bUwnuXUsVtUYDEenZZEF+QcABBK6YPAeQdNg+j7Liv4VfJX UwCTNtGWykuq1SpvuONFSwiddML7oRkhNELvDT3KM0WQ7DcYvejm7SHGjzw06OCs/pOF uXpNxrWzEd2lEVf+hM1HaRqnvtGe+F97FIWaU2G073LBJJSHgwgoI6FB6xHQhMddLb77 WevOOj51aY2dT3S9GSVWQGJX7LT3d9DrkPgRVMRsviqWEixl3RcIF2jQ5dgxa6J3HWK9 eLLK+MredZWWLudxj+bf7HVrS9tlBQmwbBmsfr++rLlCIVLW3AXfKP7u1JIUwsB3RUO2 3g+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985966; x=1740590766; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1wj/NWWDbHBPUQsB02XJgoAK7FnxuSmVDgGIKejleZ4=; b=WfA6z+OSMENvTZW+Lpy4vXiZQO21r2ugXq2Py661ASbJGJ74cPzTNWbkjPYuH3ogA0 ybnZsE9NSfvwKXNt8upD65krZKBB3GG23HgOSgO7XIp+xg/iYm2RqJ2dDAVLtJudvW1C g3P1n+0ZASkwPU77F9IsSBkDvEu+31F/BB95Ux9gauLNXy888J96XtkhD+DxKvOMGhVi c5/Gn7SIAKWhOc8OftJYqGffRyXJB38D3OGKB1s49HX++7TIa7OZM9YbXIRUNc4/TnVM zXA9xtZq0VVupnhH+E5I6sdGqswgQt7X6X4BxUE17P4ffnxmrdt/VGkIeupTx63HwMIM tmrg== X-Gm-Message-State: AOJu0YysCyZy0sQb7zrmC30IWIEB5D5vFLjUwamA8cdT7p8NST0A3Dp4 0802kCi2jkc6+3//5RwalTvvSll/MCj4NzV/Cv14lh8xv1X04xcE4VE+DHvj99zvITBQdz+jh2I L X-Gm-Gg: ASbGncvecQUFzO/e/D5DSvFbL7A0uurQm/5Mx6CdApRPGUVDW2Z8jkHHcWSh+dXWwKl izXLGV9GzoIA0zRKqa3zoVmd/79LX1FB932AgdYDqYgH6OjRwlip5UazlqjD2YNmxNgW80050GI 0Jr9j7kFLYNIgvBbG+ONv+w+/ojdVkbuTuC9TSVkkw4z+HF6lxMM0IuAnbBR7oQ67kX3j3QqHmq GZJisuqzTSO9DkLB/VWRk+EngzJBJAkhHEOu5KiOMne8rnv0iLkrFdUT3jg3g/OtbL1t8dA5gG0 AvxAda+OiKzpbBSqAqw= X-Google-Smtp-Source: AGHT+IFtaL9/XuB8qrh/MkYo4ESV+gFWdA1YcEjvsmOMak+8t8dipfJF4xY48fkZWILi3KWxQmI7cA== X-Received: by 2002:a05:6602:6d8d:b0:855:b5fe:3fb7 with SMTP id ca18e2360f4ac-855b5fe4041mr283067439f.7.1739985966274; Wed, 19 Feb 2025 09:26:06 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:05 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 2/5] eventpoll: abstract out ep_try_send_events() helper Date: Wed, 19 Feb 2025 10:22:25 -0700 Message-ID: <20250219172552.1565603-3-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation for reusing this helper in another epoll setup helper, abstract it out. Signed-off-by: Jens Axboe --- fs/eventpoll.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 565bf451df82..14466765b85d 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -1980,6 +1980,22 @@ static int ep_autoremove_wake_function(struct wait_queue_entry *wq_entry, return ret; } +static int ep_try_send_events(struct eventpoll *ep, + struct epoll_event __user *events, int maxevents) +{ + int res; + + /* + * Try to transfer events to user space. In case we get 0 events and + * there's still timeout left over, we go trying again in search of + * more luck. + */ + res = ep_send_events(ep, events, maxevents); + if (res > 0) + ep_suspend_napi_irqs(ep); + return res; +} + /** * ep_poll - Retrieves ready events, and delivers them to the caller-supplied * event buffer. @@ -2031,17 +2047,9 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, while (1) { if (eavail) { - /* - * Try to transfer events to user space. In case we get - * 0 events and there's still timeout left over, we go - * trying again in search of more luck. - */ - res = ep_send_events(ep, events, maxevents); - if (res) { - if (res > 0) - ep_suspend_napi_irqs(ep); + res = ep_try_send_events(ep, events, maxevents); + if (res) return res; - } } if (timed_out) From patchwork Wed Feb 19 17:22:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982573 Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10DFE214A6A for ; Wed, 19 Feb 2025 17:26:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985970; cv=none; b=dVjOZrOxggFJbT9wxmLcdhLLp+Jk3LmTrQA4CnUg8HXgLIPIxxNypp21e1Cze9k6MDKFKcULwCSCMsq8hQ+PnaRocqQEv5n6v1sKz43iLhHfnUiG2fITQoN3p35NWQg90kmABv4krplc6/IZtv0qKkNiEQ/lEvXh76m/u5GAxjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985970; c=relaxed/simple; bh=N/K1wXHto9EUav/plQ5H1Ct3sazV7rp7nIQXrrPuZM0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pFEtpN2fCiFH/7Zn30oaydtn1cfUSKa4yt2SzX5tBsv7YcmnKYHNfR/2kCZeSIUuE6aSTeCmhd/ApJIpTzmjGzPrk4biaflIEms9slsR0QPz2vkxd7vu8zjmP253e8PmL0eZTM9aJGerTxdheGoKrohXxUBydq4jA4cdl+ArNp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=mTgOMfe9; arc=none smtp.client-ip=209.85.166.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="mTgOMfe9" Received: by mail-io1-f54.google.com with SMTP id ca18e2360f4ac-85595f813caso1799639f.3 for ; Wed, 19 Feb 2025 09:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985967; x=1740590767; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3hLS+fhEeJItsy0qr9n/8T/ts3J225prk6HY4IC/ZoU=; b=mTgOMfe9wwXFATa4JuF9Fbk7zmVjBpQzVsvBlES10VGQAlzPC7PoNgM/egWxRo6WZW EOKGho6Inbo4gIRdRskyfHIVBOg+4PhyiKRQSQjKdSWEhjsH+Uq03XsMDac7U22VvZVe w1M+JF7IK3/DXHSXBjopLOv3aP3o2np3Qsoq7zkMHs9bKwv/uSUczFWa5sru7pTAgS9Y bUy0eXXxy+r6ZsN4KuBHpKcRvzBJ5Fwi5UbcLHtH4nm8DjsfIJfzFIFa0K3ku0LJkLUu YInwP9Wei2royHhI0knm92Th6pF1WzotToboXK4h8X369OlUm2DM1vw/Yy9/3+dc8ldW I6jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985967; x=1740590767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3hLS+fhEeJItsy0qr9n/8T/ts3J225prk6HY4IC/ZoU=; b=tdMd9RjHd8iESWqGRYaUhmChChd0fgF4qJsmqmQiMPglyZjdIMfJMvdAqZ+15nI87h QXE7Yu+DK0PXJJDSnBZtObQ41iXUIL5bGvMJ9VlbKvgS8fiOVVd1VlNqAMk3wQQ2lcbS O0Z33BJzBpv8docrHtQgqO/6FSed3ZqD7J6W3sdKRBaf9dj5CytXqkLNOtBGQDEhrofk dTbQ+i7/So8RJsM5SmzhSeBElXrwOo5qEkp648uJq0T/cD/Kn51TBSHU7Ve9q5XR/Tg2 wauIUHpmGMwXEsEYolNXURo2c9/Fv2aarrjT4HZ/rUL+PtaBQy1Owx0SDq4bLbthtBKM e3Eg== X-Gm-Message-State: AOJu0YxWsM6vmHxNH2uZsBVO4+tEy42fw4vKmSN5k5CsVL9hBF97xJTU ekDN08R6POUuaAT7/KQ7qmDonVh+yUZAVyi8wuIVCartweEKkwau22BMW+XrpkNN7qFuuM+PDZC D X-Gm-Gg: ASbGncuj8zz1pT8WEIohnx6l8A1koFiB2nSHgehYGotweVOsgGp9b0riyDyR7MzAFau fzlyzHIMScZEt4gGF52AIrR2IEDwuYLc22g5G72l4pAxlA3NJ94rFgamKtM+HJCeJbTLeg9/3V0 C1S283HfttLL8gacejPPp49Wno+iJ3h7KU0O4TOWM9yPYCaA7AyFLx8TYTCUULm68sftq12CiF9 kH1ePosQx/gCr0XKeyWAMlsb5g8Up1UWnW2Q/UKu8MeDPrn1IDTrR636sWfK7WJCmG90rqGDiFO IYu0EDViRHRqrJiM8mU= X-Google-Smtp-Source: AGHT+IFu6pkBHN/jzNR4Yx3waXcQL0mHGqKfBL7pK3ptWaTbpFR05iOXURlc6k5WEDumbHgigo/mcQ== X-Received: by 2002:a05:6602:150f:b0:855:a4a4:a938 with SMTP id ca18e2360f4ac-855a4a4a98bmr863821939f.2.1739985967617; Wed, 19 Feb 2025 09:26:07 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:06 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 3/5] eventpoll: add epoll_sendevents() helper Date: Wed, 19 Feb 2025 10:22:26 -0700 Message-ID: <20250219172552.1565603-4-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Basic helper that copies ready events to the specified userspace address. The event checking is quick and racy, it's up to the caller to ensure it retries appropriately in case 0 events are copied. Signed-off-by: Jens Axboe --- fs/eventpoll.c | 20 ++++++++++++++++++++ include/linux/eventpoll.h | 4 ++++ 2 files changed, 24 insertions(+) diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 14466765b85d..94b87aaad0f6 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2474,6 +2474,26 @@ static int ep_check_params(struct file *file, struct epoll_event __user *evs, return 0; } +int epoll_sendevents(struct file *file, struct epoll_event __user *events, + int maxevents) +{ + struct eventpoll *ep; + int ret; + + ret = ep_check_params(file, events, maxevents); + if (unlikely(ret)) + return ret; + + ep = file->private_data; + /* + * Racy call, but that's ok - it should get retried based on + * poll readiness anyway. + */ + if (ep_events_available(ep)) + return ep_try_send_events(ep, events, maxevents); + return 0; +} + /* * Implement the event wait interface for the eventpoll file. It is the kernel * part of the user space epoll_wait(2). diff --git a/include/linux/eventpoll.h b/include/linux/eventpoll.h index 0c0d00fcd131..ccb478eb174b 100644 --- a/include/linux/eventpoll.h +++ b/include/linux/eventpoll.h @@ -25,6 +25,10 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, unsigned long t /* Used to release the epoll bits inside the "struct file" */ void eventpoll_release_file(struct file *file); +/* Copy ready events to userspace */ +int epoll_sendevents(struct file *file, struct epoll_event __user *events, + int maxevents); + /* * This is called from inside fs/file_table.c:__fput() to unlink files * from the eventpoll interface. We need to have this facility to cleanup From patchwork Wed Feb 19 17:22:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982574 Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C93E214A72 for ; Wed, 19 Feb 2025 17:26:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985971; cv=none; b=XgSTjZOimHBJ0FspVsICWV3wpm4usYbUnFGsnOSHOKaBCzex/RtRSOWaNaAWlsDO/Cwm1fYFyXFIIMHyUvKjeBlwWzgcxJ7HkhlOsHju7MJEf22b6Cs5MQoIJ5M1z2cqUcSD8WYD1MmdhCExScZN1Y2Urnn72ucVrbXvFi8BbLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985971; c=relaxed/simple; bh=NKhQ3CZ0sAQuduklLkJ8jHQS9JJUaLrLvLnUNJYM5Q4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WqKsnEz78VVyrC6dIFDsdCCNiMriIbSSfNajOnqukKFgaA4Vdx7bIjuSx+vfE9ZXUAe/JkVNJJ2hajlHeKAOqOQ+/s3h49T4kwVpW8nBuKe1bbPsgGwgFp8rhV+glueEbYEw3si0+XPaMWBkPYvXtCMdvFX3wLfhaq2yt3qoaUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=uv3omP9v; arc=none smtp.client-ip=209.85.166.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="uv3omP9v" Received: by mail-io1-f47.google.com with SMTP id ca18e2360f4ac-852050432a8so1959639f.1 for ; Wed, 19 Feb 2025 09:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985969; x=1740590769; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PqeZNiF4C8ajF9Httgn8ehWaoVM+wUVbzJe1YYgdf+w=; b=uv3omP9vMBs1OH6zr+j8x1E00QLeFQ1bVFZ/3Sr5KU6i2BpRk3+NXpoiTGMEV4e47L TGLPRJqejz24lFFSiqJSNRk8hDz6qSiC8idLdZCwB7bDgKq5Re3jpQCEK8Hc072aUZy6 PgxybNO9fcyOmMJ19KKCJgnmbUWHbuVENCJk2uiQU96ormD9FUpuxIBykblBO3XQT8MM 9/rpTR9GfNKw2+YWYdL9I7M5ss4Knv4hqDblinj7hYPA+LLq5bF3jLvs5zrMdTQLzMkA /5/Si+4dSsFxlyYLIqUWv6OfKL2+w9/VkCmNQTD2p/LnSeNF+0PR+O8buTjSrILX/0FD /ZFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985969; x=1740590769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PqeZNiF4C8ajF9Httgn8ehWaoVM+wUVbzJe1YYgdf+w=; b=VkeQOXJWev6DFdphqYVKf2A1ttxpiTAPtPjLMN5R0krK+AoJa1WkMG5lbSDI/9Odi6 aCsOGMUVBsCwx3vpCabMWYopIW2uq+4df9u1fWdL5E0r2V0CfXvNWsDMFa2+3KBZkzrE i3ECtLtKCGd0nXN48DrT88rNqeIJBBgTll4v5A3TmAPgz7BPcdGUGuFCcPTF/Or/Kytq lSn/CeclbZaAaPm5T7G/Uy9PAgIocYTUOLaPPozQp5DdYlz+Is19bJS4262YFdA7bbKO 3Wfwi9t+r9T2pFbD68QE83lewKN0MZDrZ58sSk/4LaBpYllFtjr5A2xAu4lZ4jbW8ME8 w9mQ== X-Gm-Message-State: AOJu0Yw4AE0cwTFNgjXgd3845V4kTUCubvz9qUp9aLtEf0QvIMmSKeXQ Z01VWZGcB/4tFr4isCwDVyAw7gqe3uwWViEb1QYmZp+oRjtNv6UHFH00SoYObga6/SJcBUb9vva i X-Gm-Gg: ASbGnctIJUUfO9gtzMLocIaTn0pE8umVfpeisKVN9sKkkmpCJxdhViPrdZxuA7UxynF Ne22GIjBfYRSkIw04phDIxaPjCIt/PNfg2Nq+d6p3XXHKMJzcWQrT4oIsyauG3FeUSGS3dskhEN dnkQt2sdCUXUVLFvemao5E2bZL/entkrYcxM0I6E6nPHO0eLdLub9m/b5d/PfG4X5Lu19misgYm D5ZmxaPeF2IHtV8F3CLFYrD9jSBCs3oUTO53etu99k1rcmafhK/PJUM+4+bJDY5HW4hcJ4XTxxQ m5HMOTfwlKJzVzT32u4= X-Google-Smtp-Source: AGHT+IHRVFWWS5YvlkaNF1Q/MDG4uwtiTei/nIZPfpHN+WkrFtLSjd0dVoryMXxuADhDSJM8pIGxUA== X-Received: by 2002:a05:6602:6d87:b0:855:b0eb:3fe6 with SMTP id ca18e2360f4ac-855c32ce3c0mr10130239f.8.1739985968846; Wed, 19 Feb 2025 09:26:08 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:08 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 4/5] io_uring/epoll: remove CONFIG_EPOLL guards Date: Wed, 19 Feb 2025 10:22:27 -0700 Message-ID: <20250219172552.1565603-5-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Just have the Makefile add the object if epoll is enabled, then it's not necessary to guard the entire epoll.c file inside an CONFIG_EPOLL ifdef. Signed-off-by: Jens Axboe --- io_uring/Makefile | 9 +++++---- io_uring/epoll.c | 2 -- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/io_uring/Makefile b/io_uring/Makefile index 98e48339d84d..3e28a741ca15 100644 --- a/io_uring/Makefile +++ b/io_uring/Makefile @@ -11,10 +11,11 @@ obj-$(CONFIG_IO_URING) += io_uring.o opdef.o kbuf.o rsrc.o notif.o \ eventfd.o uring_cmd.o openclose.o \ sqpoll.o xattr.o nop.o fs.o splice.o \ sync.o msg_ring.o advise.o openclose.o \ - epoll.o statx.o timeout.o fdinfo.o \ - cancel.o waitid.o register.o \ - truncate.o memmap.o alloc_cache.o + statx.o timeout.o fdinfo.o cancel.o \ + waitid.o register.o truncate.o \ + memmap.o alloc_cache.o obj-$(CONFIG_IO_URING_ZCRX) += zcrx.o obj-$(CONFIG_IO_WQ) += io-wq.o obj-$(CONFIG_FUTEX) += futex.o -obj-$(CONFIG_NET_RX_BUSY_POLL) += napi.o +obj-$(CONFIG_EPOLL) += epoll.o +obj-$(CONFIG_NET_RX_BUSY_POLL) += napi.o diff --git a/io_uring/epoll.c b/io_uring/epoll.c index 89bff2068a19..7848d9cc073d 100644 --- a/io_uring/epoll.c +++ b/io_uring/epoll.c @@ -12,7 +12,6 @@ #include "io_uring.h" #include "epoll.h" -#if defined(CONFIG_EPOLL) struct io_epoll { struct file *file; int epfd; @@ -58,4 +57,3 @@ int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags) io_req_set_res(req, ret, 0); return IOU_OK; } -#endif From patchwork Wed Feb 19 17:22:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13982575 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 988A5214A81 for ; Wed, 19 Feb 2025 17:26:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985973; cv=none; b=Ey81UPqsccwlYEdIbdyODNAYGmPsPW5eKMb7GK0k1itARBqlcZqSfy5fFy33nPs+597Sh07NZFqP8/ymoGZspGodfVidfa1qjTWd8R2iNjtcKz43zy9AJRtnyLQcd4bICWeB21+JRSQM4WO0SDIBPffGWLFXM/22dvYhkRDhENM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739985973; c=relaxed/simple; bh=Dgu/2bYkWXrAMq3iF1/8GI7X79S6G6gmhyLC1ec2mfY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U+ccJkm5/MlLYPaT8v3J0tAlMW52lQXLjrwHf1weSctv0kIneUE56C9524U85VsWN+j/dBWm7ow7qWqDiW2AOgeBTRkJ7X0GXs4sR0MtPcJkuCzI7vuDKXJbDf97a/piZERJmriVKt++tbTing9B43sGWR3UVjKikwqB/bZHwQc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=C2nFRDgx; arc=none smtp.client-ip=209.85.166.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="C2nFRDgx" Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-855a7e3be0eso1088139f.1 for ; Wed, 19 Feb 2025 09:26:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1739985970; x=1740590770; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1gAHZg9kSYTvKQ33H52aoGebUTskPe032Qb9P2hQu70=; b=C2nFRDgxkeUJxOFus0XdLUwXUYVZZEsSkdAI1XHXzqRNSSQQmrJD9OXmqK81dXZ+BU niwem1/kEPxvdxUZ4YiskDmNk3XSt7OfAbpvEKkdN7ArTzI6ot1FF25GTJ5yH7Pn9+uZ flvqOxHUsldj7pQwY3oAWiOs9wBRDD647+RID/uADI4JAAEs46G5JA7AsA7MMk5HErtN XhWEQDxMA0mD9Ygz5zU2DfKt8w2L6YiZuftaXcoJiiDHarie7tDMknHTp/GULPB2F199 xFwnXfySPzu1SCSPP34lIziFl53ES2XTG+iMHkehQXmKmPF++FLWcvkUio6s1clC0SOr tiAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739985970; x=1740590770; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1gAHZg9kSYTvKQ33H52aoGebUTskPe032Qb9P2hQu70=; b=pSdXA7SZCYxEhMTxNxkhAiQGXWKxoSZ0kgUym828aSW3SkGT8OcC3R4j5juSBSihi2 //ezB5TDt4PbZFbpkYxgFJcUwld48gspcCnkk/Z58jDOFfaK72GdhCEC00vRYI0XS8Zm SpVDnzCWlbbYnLrC5mZ6XZy9eorQ9fZI2GMHikDwBTH5D1Q02sD1GotPKfp9R0RK/1iT IVPdvPs9piQb9xXLw5dX8f+Y72V9PpYCnbrAzohoJzwnfx5GX/nukK4u/NH5+uSDU90b zKANzGirliXWOfXmyqtOYm346gTasSOpQ3ujp541wIgxe267N7FKjsHGcdPZgbGYiYcs 7Kgg== X-Gm-Message-State: AOJu0Yy0aIo9WeGuCGTJT8nexDI2NspzzJQD62FckBs3DNlaxBLrexOj +XmIcE9OtAROLQdphIHrmyjazrCopkaeGYG8C96GA+Bhpk9Rx5GMhNJMpFq+8Fu+tHAZH0/uE8T a X-Gm-Gg: ASbGncs5ngFQg5NXtcG2Zlz/ESNcDlXuTVjGfhjVJ1se4buLgApN0r0sJoj+RbGIwCM A16z5QDYqJgPJuQfwp+YtWf04a1bsCavnCAwPvVOOzY8iDgO5DJ8SmtK5QYacwpkrSiNKel5CCG VMIDKTE3KVxnrrUN7/2IO3ISsDjIY6S6e7NvoDmUxiEx2rZxUnvDpZTUjktPCJ2GDtsbG3tVma5 iWl8aYkTL3g7WqwnKRFg+KymNmlboZ+r5gzUlxMY9ALWGF/5HfTm8MU5U3q/sRxvC29YXMHEtXI uaCYVRVmf6I3RhBKKSU= X-Google-Smtp-Source: AGHT+IHjnt3iA5lsKLTSE6k9kN1FP0fHcpHTu6e9TZVmjZOfhdBtx2VkZxFxTUM+se6BvwdrScUk8Q== X-Received: by 2002:a05:6602:6d8d:b0:855:b5fe:3fb7 with SMTP id ca18e2360f4ac-855b5fe4041mr283100139f.7.1739985970155; Wed, 19 Feb 2025 09:26:10 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id ca18e2360f4ac-8558f3ccdcesm142192839f.16.2025.02.19.09.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 09:26:09 -0800 (PST) From: Jens Axboe To: io-uring@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, brauner@kernel.org, asml.silence@gmail.com, Jens Axboe Subject: [PATCH 5/5] io_uring/epoll: add support for IORING_OP_EPOLL_WAIT Date: Wed, 19 Feb 2025 10:22:28 -0700 Message-ID: <20250219172552.1565603-6-axboe@kernel.dk> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250219172552.1565603-1-axboe@kernel.dk> References: <20250219172552.1565603-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For existing epoll event loops that can't fully convert to io_uring, the used approach is usually to add the io_uring fd to the epoll instance and use epoll_wait() to wait on both "legacy" and io_uring events. While this work, it isn't optimal as: 1) epoll_wait() is pretty limited in what it can do. It does not support partial reaping of events, or waiting on a batch of events. 2) When an io_uring ring is added to an epoll instance, it activates the io_uring "I'm being polled" logic which slows things down. Rather than use this approach, with EPOLL_WAIT support added to io_uring, event loops can use the normal io_uring wait logic for everything, as long as an epoll wait request has been armed with io_uring. Note that IORING_OP_EPOLL_WAIT does NOT take a timeout value, as this is an async request. Waiting on io_uring events in general has various timeout parameters, and those are the ones that should be used when waiting on any kind of request. If events are immediately available for reaping, then This opcode will return those immediately. If none are available, then it will post an async completion when they become available. cqe->res will contain either an error code (< 0 value) for a malformed request, invalid epoll instance, etc. It will return a positive result indicating how many events were reaped. IORING_OP_EPOLL_WAIT requests may be canceled using the normal io_uring cancelation infrastructure. Signed-off-by: Jens Axboe --- include/uapi/linux/io_uring.h | 1 + io_uring/epoll.c | 33 +++++++++++++++++++++++++++++++++ io_uring/epoll.h | 2 ++ io_uring/opdef.c | 14 ++++++++++++++ 4 files changed, 50 insertions(+) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 05d6255b0f6a..135eb9296296 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -280,6 +280,7 @@ enum io_uring_op { IORING_OP_BIND, IORING_OP_LISTEN, IORING_OP_RECV_ZC, + IORING_OP_EPOLL_WAIT, /* this goes last, obviously */ IORING_OP_LAST, diff --git a/io_uring/epoll.c b/io_uring/epoll.c index 7848d9cc073d..6d2c48ba1923 100644 --- a/io_uring/epoll.c +++ b/io_uring/epoll.c @@ -20,6 +20,12 @@ struct io_epoll { struct epoll_event event; }; +struct io_epoll_wait { + struct file *file; + int maxevents; + struct epoll_event __user *events; +}; + int io_epoll_ctl_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_epoll *epoll = io_kiocb_to_cmd(req, struct io_epoll); @@ -57,3 +63,30 @@ int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags) io_req_set_res(req, ret, 0); return IOU_OK; } + +int io_epoll_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) +{ + struct io_epoll_wait *iew = io_kiocb_to_cmd(req, struct io_epoll_wait); + + if (sqe->off || sqe->rw_flags || sqe->buf_index || sqe->splice_fd_in) + return -EINVAL; + + iew->maxevents = READ_ONCE(sqe->len); + iew->events = u64_to_user_ptr(READ_ONCE(sqe->addr)); + return 0; +} + +int io_epoll_wait(struct io_kiocb *req, unsigned int issue_flags) +{ + struct io_epoll_wait *iew = io_kiocb_to_cmd(req, struct io_epoll_wait); + int ret; + + ret = epoll_sendevents(req->file, iew->events, iew->maxevents); + if (ret == 0) + return -EAGAIN; + if (ret < 0) + req_set_fail(req); + + io_req_set_res(req, ret, 0); + return IOU_OK; +} diff --git a/io_uring/epoll.h b/io_uring/epoll.h index 870cce11ba98..4111997c360b 100644 --- a/io_uring/epoll.h +++ b/io_uring/epoll.h @@ -3,4 +3,6 @@ #if defined(CONFIG_EPOLL) int io_epoll_ctl_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); int io_epoll_ctl(struct io_kiocb *req, unsigned int issue_flags); +int io_epoll_wait_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe); +int io_epoll_wait(struct io_kiocb *req, unsigned int issue_flags); #endif diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 89f50ecadeaf..9344534780a0 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -527,6 +527,17 @@ const struct io_issue_def io_issue_defs[] = { .issue = io_recvzc, #else .prep = io_eopnotsupp_prep, +#endif + }, + [IORING_OP_EPOLL_WAIT] = { + .needs_file = 1, + .audit_skip = 1, + .pollin = 1, +#if defined(CONFIG_EPOLL) + .prep = io_epoll_wait_prep, + .issue = io_epoll_wait, +#else + .prep = io_eopnotsupp_prep, #endif }, }; @@ -761,6 +772,9 @@ const struct io_cold_def io_cold_defs[] = { [IORING_OP_RECV_ZC] = { .name = "RECV_ZC", }, + [IORING_OP_EPOLL_WAIT] = { + .name = "EPOLL_WAIT", + }, }; const char *io_uring_get_opcode(u8 opcode)