From patchwork Fri Jul 10 14:19:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 11656777 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45A5813B4 for ; Fri, 10 Jul 2020 14:20:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 201082085B for ; Fri, 10 Jul 2020 14:20:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AlJpxDdx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728208AbgGJOUH (ORCPT ); Fri, 10 Jul 2020 10:20:07 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35247 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727828AbgGJOUG (ORCPT ); Fri, 10 Jul 2020 10:20:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594390805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iINrqTduzdzlg8cDRyIDlpEwPnwdLZ0XaojMAYpWNSc=; b=AlJpxDdxO3RZDcaylVpz7sqmu9eXZHQF7Hs0+YDx5NnE36hCPDOy5XiuwL/0aV/3P12nRS 1cW1OsPOxdvL8rS5sL/9yIkAwpLZGl7CPSz0Qsw83vlWVDEMU0u8cLg/eJZeb+XU0O/91E Khn0gMWYKj+dviB7vKzAzbsw2NIumCg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-219-h_9ogIOyOk-t4fP_p4hujw-1; Fri, 10 Jul 2020 10:20:00 -0400 X-MC-Unique: h_9ogIOyOk-t4fP_p4hujw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 435AF1B18BC7; Fri, 10 Jul 2020 14:19:59 +0000 (UTC) Received: from steredhat.redhat.com (ovpn-112-4.ams2.redhat.com [10.36.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 787C55C5B7; Fri, 10 Jul 2020 14:19:55 +0000 (UTC) From: Stefano Garzarella To: Jens Axboe Cc: Sargun Dhillon , Kees Cook , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Kernel Hardening , Jann Horn , Aleksa Sarai , Christian Brauner , Stefan Hajnoczi , io-uring@vger.kernel.org, Alexander Viro , Jeff Moyer Subject: [PATCH RFC 1/3] io_uring: use an enumeration for io_uring_register(2) opcodes Date: Fri, 10 Jul 2020 16:19:43 +0200 Message-Id: <20200710141945.129329-2-sgarzare@redhat.com> In-Reply-To: <20200710141945.129329-1-sgarzare@redhat.com> References: <20200710141945.129329-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The enumeration allows us to keep track of the last io_uring_register(2) opcode available. Behaviour and opcodes names don't change. Signed-off-by: Stefano Garzarella --- include/uapi/linux/io_uring.h | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 92c22699a5a7..2d18f1d0b5df 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -252,17 +252,22 @@ struct io_uring_params { /* * io_uring_register(2) opcodes and arguments */ -#define IORING_REGISTER_BUFFERS 0 -#define IORING_UNREGISTER_BUFFERS 1 -#define IORING_REGISTER_FILES 2 -#define IORING_UNREGISTER_FILES 3 -#define IORING_REGISTER_EVENTFD 4 -#define IORING_UNREGISTER_EVENTFD 5 -#define IORING_REGISTER_FILES_UPDATE 6 -#define IORING_REGISTER_EVENTFD_ASYNC 7 -#define IORING_REGISTER_PROBE 8 -#define IORING_REGISTER_PERSONALITY 9 -#define IORING_UNREGISTER_PERSONALITY 10 +enum { + IORING_REGISTER_BUFFERS, + IORING_UNREGISTER_BUFFERS, + IORING_REGISTER_FILES, + IORING_UNREGISTER_FILES, + IORING_REGISTER_EVENTFD, + IORING_UNREGISTER_EVENTFD, + IORING_REGISTER_FILES_UPDATE, + IORING_REGISTER_EVENTFD_ASYNC, + IORING_REGISTER_PROBE, + IORING_REGISTER_PERSONALITY, + IORING_UNREGISTER_PERSONALITY, + + /* this goes last */ + IORING_REGISTER_LAST +}; struct io_uring_files_update { __u32 offset; From patchwork Fri Jul 10 14:19:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 11656781 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 996CA13B4 for ; Fri, 10 Jul 2020 14:20:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 770F22078B for ; Fri, 10 Jul 2020 14:20:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gFu0cLjE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728348AbgGJOUQ (ORCPT ); Fri, 10 Jul 2020 10:20:16 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:29994 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728182AbgGJOUP (ORCPT ); Fri, 10 Jul 2020 10:20:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594390812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S6cQhuOcDdhnVYpuDpB8H3qcMEL452aYxMh3H44NLcU=; b=gFu0cLjE/IQiYuafn8GAAu8t37cZ/7ujlcnm/bO4ZdFwu/9G7cYbYkimrQ9qgJimYWIDvi tK8UM00gx0C22ufypo+6iuaMZJPNWuJ46jb6FRRvp05NYS8zk0jvMb4vEnqe0pcah+/iuL pSebH7FPMMH26GSOKSBUCq3QxQw4fEE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-273-vMgFKFOeON6292laPskXsQ-1; Fri, 10 Jul 2020 10:20:08 -0400 X-MC-Unique: vMgFKFOeON6292laPskXsQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A33B78005B0; Fri, 10 Jul 2020 14:20:06 +0000 (UTC) Received: from steredhat.redhat.com (ovpn-112-4.ams2.redhat.com [10.36.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB4795C3F8; Fri, 10 Jul 2020 14:19:59 +0000 (UTC) From: Stefano Garzarella To: Jens Axboe Cc: Sargun Dhillon , Kees Cook , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Kernel Hardening , Jann Horn , Aleksa Sarai , Christian Brauner , Stefan Hajnoczi , io-uring@vger.kernel.org, Alexander Viro , Jeff Moyer Subject: [PATCH RFC 2/3] io_uring: add IOURING_REGISTER_RESTRICTIONS opcode Date: Fri, 10 Jul 2020 16:19:44 +0200 Message-Id: <20200710141945.129329-3-sgarzare@redhat.com> In-Reply-To: <20200710141945.129329-1-sgarzare@redhat.com> References: <20200710141945.129329-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The new io_uring_register(2) IOURING_REGISTER_RESTRICTIONS opcode permanently installs a feature whitelist on an io_ring_ctx. The io_ring_ctx can then be passed to untrusted code with the knowledge that only operations present in the whitelist can be executed. The whitelist approach ensures that new features added to io_uring do not accidentally become available when an existing application is launched on a newer kernel version. Currently is it possible to restrict sqe opcodes and register opcodes. It is also possible to allow only fixed files. IOURING_REGISTER_RESTRICTIONS can only be made once. Afterwards it is not possible to change restrictions anymore. This prevents untrusted code from removing restrictions. Suggested-by: Stefan Hajnoczi Signed-off-by: Stefano Garzarella --- fs/io_uring.c | 98 ++++++++++++++++++++++++++++++++++- include/uapi/linux/io_uring.h | 30 +++++++++++ 2 files changed, 127 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index d37d7ea5ebe5..4768a9973d4b 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -218,6 +218,13 @@ struct io_buffer { __u16 bid; }; +struct io_restriction { + DECLARE_BITMAP(register_op, IORING_REGISTER_LAST); + DECLARE_BITMAP(sqe_op, IORING_OP_LAST); + DECLARE_BITMAP(restriction_op, IORING_RESTRICTION_LAST); + bool enabled; /* TODO: remove and use a flag ?? */ +}; + struct io_ring_ctx { struct { struct percpu_ref refs; @@ -337,6 +344,7 @@ struct io_ring_ctx { struct llist_head file_put_llist; struct work_struct exit_work; + struct io_restriction restrictions; }; /* @@ -5491,6 +5499,11 @@ static int io_req_set_file(struct io_submit_state *state, struct io_kiocb *req, if (unlikely(!fixed && io_async_submit(req->ctx))) return -EBADF; + if (unlikely(!fixed && req->ctx->restrictions.enabled && + test_bit(IORING_RESTRICTION_FIXED_FILES_ONLY, + req->ctx->restrictions.restriction_op))) + return -EACCES; + return io_file_get(state, req, fd, &req->file, fixed); } @@ -5895,6 +5908,10 @@ static int io_init_req(struct io_ring_ctx *ctx, struct io_kiocb *req, if (unlikely(req->opcode >= IORING_OP_LAST)) return -EINVAL; + if (unlikely(ctx->restrictions.enabled && + !test_bit(req->opcode, ctx->restrictions.sqe_op))) + return -EACCES; + if (unlikely(io_sq_thread_acquire_mm(ctx, req))) return -EFAULT; @@ -8079,6 +8096,69 @@ static int io_unregister_personality(struct io_ring_ctx *ctx, unsigned id) return -EINVAL; } +static int io_register_restrictions(struct io_ring_ctx *ctx, void __user *arg, + unsigned int nr_args) +{ + struct io_uring_restriction *res; + size_t size; + int i, ret; + + /* We allow only a single restrictions registration */ + if (ctx->restrictions.enabled) + return -EINVAL; /* TODO: check ret value */ + + /* TODO: Is it okay to set a maximum? */ + if (!arg || nr_args > 256) + return -EINVAL; + + size = array_size(nr_args, sizeof(*res)); + if (size == SIZE_MAX) + return -EOVERFLOW; + + res = kmalloc(size, GFP_KERNEL); + if (!res) + return -ENOMEM; + + if (copy_from_user(res, arg, size)) { + ret = -EFAULT; + goto out; + } + + for (i = 0; i < nr_args; i++) { + if (res[i].opcode >= IORING_RESTRICTION_LAST) { + ret = -EINVAL; + goto out; + } + + __set_bit(res[i].opcode, ctx->restrictions.restriction_op); + + if (res[i].opcode == IORING_RESTRICTION_REGISTER_OP) { + if (res[i].register_op >= IORING_REGISTER_LAST) { + ret = -EINVAL; + goto out; + } + + __set_bit(res[i].register_op, + ctx->restrictions.register_op); + } else if (res[i].opcode == IORING_RESTRICTION_SQE_OP) { + if (res[i].sqe_op >= IORING_OP_LAST) { + ret = -EINVAL; + goto out; + } + + __set_bit(res[i].sqe_op, ctx->restrictions.sqe_op); + } + } + + ctx->restrictions.enabled = true; + + ret = 0; +out: + /* TODO: should we reset all restrictions if an error happened? */ + kfree(res); + return ret; +} + static bool io_register_op_must_quiesce(int op) { switch (op) { @@ -8125,6 +8205,18 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, if (ret) { percpu_ref_resurrect(&ctx->refs); ret = -EINTR; + goto out_quiesce; + } + } + + if (ctx->restrictions.enabled) { + if (opcode >= IORING_REGISTER_LAST) { + ret = -EINVAL; + goto out; + } + + if (!test_bit(opcode, ctx->restrictions.register_op)) { + ret = -EACCES; goto out; } } @@ -8188,15 +8280,19 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, break; ret = io_unregister_personality(ctx, nr_args); break; + case IORING_REGISTER_RESTRICTIONS: + ret = io_register_restrictions(ctx, arg, nr_args); + break; default: ret = -EINVAL; break; } +out: if (io_register_op_must_quiesce(opcode)) { /* bring the ctx back to life */ percpu_ref_reinit(&ctx->refs); -out: +out_quiesce: reinit_completion(&ctx->ref_comp); } return ret; diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 2d18f1d0b5df..69f4684c988d 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -264,6 +264,7 @@ enum { IORING_REGISTER_PROBE, IORING_REGISTER_PERSONALITY, IORING_UNREGISTER_PERSONALITY, + IORING_REGISTER_RESTRICTIONS, /* this goes last */ IORING_REGISTER_LAST @@ -292,4 +293,33 @@ struct io_uring_probe { struct io_uring_probe_op ops[0]; }; +struct io_uring_restriction { + __u16 opcode; + union { + __u8 register_op; /* IORING_RESTRICTION_REGISTER_OP */ + __u8 sqe_op; /* IORING_RESTRICTION_SQE_OP */ + }; + __u8 resv; + __u32 resv2[3]; +}; + +/* + * io_uring_restriction->opcode values + */ +enum { + /* Allow an io_uring_register(2) opcode */ + IORING_RESTRICTION_REGISTER_OP, + + /* Allow an sqe opcode */ + IORING_RESTRICTION_SQE_OP, + + /* Only allow fixed files */ + IORING_RESTRICTION_FIXED_FILES_ONLY, + + /* Only allow registered addresses and translate them */ + //TODO: IORING_RESTRICTION_BUFFER_CHECK, + + IORING_RESTRICTION_LAST +}; + #endif From patchwork Fri Jul 10 14:19:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Garzarella X-Patchwork-Id: 11656783 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53F1F913 for ; Fri, 10 Jul 2020 14:20:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 381E62078B for ; Fri, 10 Jul 2020 14:20:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="d6A2zLWI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728371AbgGJOUU (ORCPT ); Fri, 10 Jul 2020 10:20:20 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:24955 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728182AbgGJOUT (ORCPT ); Fri, 10 Jul 2020 10:20:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594390817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8QuqU3Vbf6mz/t3wiNDm6uN168TBBL7sMxA2awxIqYc=; b=d6A2zLWIctpe78TXsaTEFxhkxQs4nQZ0PVe25pIcFvUn5zJe6msedUZiJOEw8ZGnJE+cq1 rxAP9+Z9Ma452EWMTJZaVcmXS1nSjDmqYbat0wLS05T/eUDSzjDaLq38RUbrsCdtp9uq4L XPD8SvYHXmNzv8ZnEMzqOaBAZJgqtPA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-373-P8zfqDJoOUukHVhc33pOiA-1; Fri, 10 Jul 2020 10:20:15 -0400 X-MC-Unique: P8zfqDJoOUukHVhc33pOiA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A4991800EB6; Fri, 10 Jul 2020 14:20:13 +0000 (UTC) Received: from steredhat.redhat.com (ovpn-112-4.ams2.redhat.com [10.36.112.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 146285C1D6; Fri, 10 Jul 2020 14:20:06 +0000 (UTC) From: Stefano Garzarella To: Jens Axboe Cc: Sargun Dhillon , Kees Cook , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Kernel Hardening , Jann Horn , Aleksa Sarai , Christian Brauner , Stefan Hajnoczi , io-uring@vger.kernel.org, Alexander Viro , Jeff Moyer Subject: [PATCH RFC 3/3] io_uring: allow disabling rings during the creation Date: Fri, 10 Jul 2020 16:19:45 +0200 Message-Id: <20200710141945.129329-4-sgarzare@redhat.com> In-Reply-To: <20200710141945.129329-1-sgarzare@redhat.com> References: <20200710141945.129329-1-sgarzare@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch adds a new IORING_SETUP_R_DISABLED flag to start the rings disabled, allowing the user to register restrictions, buffers, files, before to start processing SQEs. When IORING_SETUP_R_DISABLED is set, SQE are not processed and SQPOLL kthread is not started. The restrictions registration are allowed only when the rings are disable to prevent concurrency issue while processing SQEs. The rings can be enabled using IORING_REGISTER_ENABLE_RINGS opcode with io_uring_register(2). Suggested-by: Jens Axboe Signed-off-by: Stefano Garzarella --- fs/io_uring.c | 57 ++++++++++++++++++++++++++++++----- include/uapi/linux/io_uring.h | 2 ++ 2 files changed, 51 insertions(+), 8 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 4768a9973d4b..52a75bf4206f 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -6955,8 +6955,8 @@ static int io_init_wq_offload(struct io_ring_ctx *ctx, return ret; } -static int io_sq_offload_start(struct io_ring_ctx *ctx, - struct io_uring_params *p) +static int io_sq_offload_create(struct io_ring_ctx *ctx, + struct io_uring_params *p) { int ret; @@ -6993,7 +6993,6 @@ static int io_sq_offload_start(struct io_ring_ctx *ctx, ctx->sqo_thread = NULL; goto err; } - wake_up_process(ctx->sqo_thread); } else if (p->flags & IORING_SETUP_SQ_AFF) { /* Can't have SQ_AFF without SQPOLL */ ret = -EINVAL; @@ -7012,6 +7011,18 @@ static int io_sq_offload_start(struct io_ring_ctx *ctx, return ret; } +static int io_sq_offload_start(struct io_ring_ctx *ctx) +{ + if (ctx->flags & IORING_SETUP_SQPOLL) { + if (!ctx->sqo_thread) + return -EINVAL; /* TODO: check errno */ + + wake_up_process(ctx->sqo_thread); + } + + return 0; +} + static void io_unaccount_mem(struct user_struct *user, unsigned long nr_pages) { atomic_long_sub(nr_pages, &user->locked_vm); @@ -7632,9 +7643,6 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, int submitted = 0; struct fd f; - if (current->task_works) - task_work_run(); - if (flags & ~(IORING_ENTER_GETEVENTS | IORING_ENTER_SQ_WAKEUP)) return -EINVAL; @@ -7651,6 +7659,12 @@ SYSCALL_DEFINE6(io_uring_enter, unsigned int, fd, u32, to_submit, if (!percpu_ref_tryget(&ctx->refs)) goto out_fput; + if (ctx->flags & IORING_SETUP_R_DISABLED) + return -EBADF; + + if (current->task_works) + task_work_run(); + /* * For SQ polling, the thread will do all submissions and completions. * Just return the requested submit count, and wake the thread if @@ -7956,10 +7970,16 @@ static int io_uring_create(unsigned entries, struct io_uring_params *p, if (ret) goto err; - ret = io_sq_offload_start(ctx, p); + ret = io_sq_offload_create(ctx, p); if (ret) goto err; + if (!(p->flags & IORING_SETUP_R_DISABLED)) { + ret = io_sq_offload_start(ctx); + if (ret) + goto err; + } + memset(&p->sq_off, 0, sizeof(p->sq_off)); p->sq_off.head = offsetof(struct io_rings, sq.head); p->sq_off.tail = offsetof(struct io_rings, sq.tail); @@ -8020,7 +8040,8 @@ static long io_uring_setup(u32 entries, struct io_uring_params __user *params) if (p.flags & ~(IORING_SETUP_IOPOLL | IORING_SETUP_SQPOLL | IORING_SETUP_SQ_AFF | IORING_SETUP_CQSIZE | - IORING_SETUP_CLAMP | IORING_SETUP_ATTACH_WQ)) + IORING_SETUP_CLAMP | IORING_SETUP_ATTACH_WQ | + IORING_SETUP_R_DISABLED)) return -EINVAL; return io_uring_create(entries, &p, params); @@ -8103,6 +8124,10 @@ static int io_register_restrictions(struct io_ring_ctx *ctx, void __user *arg, size_t size; int i, ret; + /* Restrictions allowed only if rings started disabled */ + if (!(ctx->flags & IORING_SETUP_R_DISABLED)) + return -EINVAL; + /* We allow only a single restrictions registration */ if (ctx->restrictions.enabled) return -EINVAL; /* TODO: check ret value */ @@ -8159,6 +8184,16 @@ static int io_register_restrictions(struct io_ring_ctx *ctx, void __user *arg, return ret; } +static int io_register_enable_rings(struct io_ring_ctx *ctx) +{ + if (!(ctx->flags & IORING_SETUP_R_DISABLED)) + return -EINVAL; + + ctx->flags &= ~IORING_SETUP_R_DISABLED; + + return io_sq_offload_start(ctx); +} + static bool io_register_op_must_quiesce(int op) { switch (op) { @@ -8280,6 +8315,12 @@ static int __io_uring_register(struct io_ring_ctx *ctx, unsigned opcode, break; ret = io_unregister_personality(ctx, nr_args); break; + case IORING_REGISTER_ENABLE_RINGS: + ret = -EINVAL; + if (arg || nr_args) + break; + ret = io_register_enable_rings(ctx); + break; case IORING_REGISTER_RESTRICTIONS: ret = io_register_restrictions(ctx, arg, nr_args); break; diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 69f4684c988d..57081c746b06 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -94,6 +94,7 @@ enum { #define IORING_SETUP_CQSIZE (1U << 3) /* app defines CQ size */ #define IORING_SETUP_CLAMP (1U << 4) /* clamp SQ/CQ ring sizes */ #define IORING_SETUP_ATTACH_WQ (1U << 5) /* attach to existing wq */ +#define IORING_SETUP_R_DISABLED (1U << 6) /* start with ring disabled */ enum { IORING_OP_NOP, @@ -265,6 +266,7 @@ enum { IORING_REGISTER_PERSONALITY, IORING_UNREGISTER_PERSONALITY, IORING_REGISTER_RESTRICTIONS, + IORING_REGISTER_ENABLE_RINGS, /* this goes last */ IORING_REGISTER_LAST