From patchwork Tue Nov 14 16:32:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13455604 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 657D03FE38 for ; Tue, 14 Nov 2023 16:33:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O9xJJH+c" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AE45134 for ; Tue, 14 Nov 2023 08:33:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1699979627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=BkORamFmgjQm3rT3wXmKJVf5l2MYL+TWgwU+R0JxK1c=; b=O9xJJH+cmHlfqT7imG4tOUTDYcg16yAZc2RMofJvynu1mumE6/NYEyh1g3gXCbQ+VgmD9d U3wNJUEs8TJhSDiFp5NatW0alpEMY3tugI8n8wZUQhBbmby2AML8mDxofV/aq+xEkjCWVs YrbkSI3nT6LY9JdGEHUvZ7PEVtD0QqM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-520-yLGSOGLEP8ui6el_IihhxQ-1; Tue, 14 Nov 2023 11:33:41 -0500 X-MC-Unique: yLGSOGLEP8ui6el_IihhxQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 49EE5811E8F; Tue, 14 Nov 2023 16:33:41 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.253]) by smtp.corp.redhat.com (Postfix) with SMTP id CA84B5028; Tue, 14 Nov 2023 16:33:39 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Tue, 14 Nov 2023 17:32:36 +0100 (CET) Date: Tue, 14 Nov 2023 17:32:34 +0100 From: Oleg Nesterov To: Alexei Starovoitov , Yonghong Song Cc: Chuyi Zhou , Daniel Borkmann , Kui-Feng Lee , linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 1/3] bpf: task_group_seq_get_next: use __next_thread() rather than next_thread() Message-ID: <20231114163234.GA890@redhat.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231114163211.GA874@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 X-Patchwork-Delegate: bpf@iogearbox.net Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong. task_group_seq_get_next() can return the group leader twice if it races with mt-thread exec which changes the group->leader's pid. Change the main loop to use __next_thread(), kill "next_tid == common->pid" check. __next_thread() can't loop forever, we can also change this code to retry if next_tid == 0. Signed-off-by: Oleg Nesterov Acked-by: Yonghong Song --- kernel/bpf/task_iter.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 26082b97894d..51ae15e2b290 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -70,15 +70,13 @@ static struct task_struct *task_group_seq_get_next(struct bpf_iter_seq_task_comm return NULL; retry: - task = next_thread(task); + task = __next_thread(task); + if (!task) + return NULL; next_tid = __task_pid_nr_ns(task, PIDTYPE_PID, common->ns); - if (!next_tid || next_tid == common->pid) { - /* Run out of tasks of a process. The tasks of a - * thread_group are linked as circular linked list. - */ - return NULL; - } + if (!next_tid) + goto retry; if (skip_if_dup_files && task->files == task->group_leader->files) goto retry; From patchwork Tue Nov 14 16:32:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13455605 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6581B3FE54 for ; Tue, 14 Nov 2023 16:33:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="G6VouqA2" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2199E132 for ; Tue, 14 Nov 2023 08:33:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1699979627; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=2Mg3UCFMr1SH7PUf+Sl5YMeVrQDgFkP/RxqlBy+YY5Y=; b=G6VouqA2KyoyP5+loUFGnXSwgNhJ+XWoK9aBkpaxtQu5afl8OiQKhe6fCjEfX2aDOMkVWf iSVm36qxrlRqs91nlDUltIkzY3CaAhF2Hhkhw0A1OmAjRJv5K9Ntfy6wFfx6rQoeA7KFTJ vlj+pvMJS9J4ZxRGurQ45pyJcmUC7aA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-140-areqUpPuN-OXmCaWVP_OIg-1; Tue, 14 Nov 2023 11:33:45 -0500 X-MC-Unique: areqUpPuN-OXmCaWVP_OIg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 24813811E93; Tue, 14 Nov 2023 16:33:44 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.253]) by smtp.corp.redhat.com (Postfix) with SMTP id 46294493113; Tue, 14 Nov 2023 16:33:42 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Tue, 14 Nov 2023 17:32:39 +0100 (CET) Date: Tue, 14 Nov 2023 17:32:37 +0100 From: Oleg Nesterov To: Alexei Starovoitov , Yonghong Song Cc: Chuyi Zhou , Daniel Borkmann , Kui-Feng Lee , linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 2/3] bpf: bpf_iter_task_next: use __next_thread() rather than next_thread() Message-ID: <20231114163237.GA897@redhat.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231114163211.GA874@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-Patchwork-Delegate: bpf@iogearbox.net Lockless use of next_thread() should be avoided, kernel/bpf/task_iter.c is the last user and the usage is wrong. bpf_iter_task_next() can loop forever, "kit->pos == kit->task" can never happen if kit->pos execs. Change this code to use __next_thread(). With or without this change the usage of kit->pos/task and next_task() doesn't look nice, see the next patch. Signed-off-by: Oleg Nesterov Acked-by: Yonghong Song --- kernel/bpf/task_iter.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 51ae15e2b290..d42e08d0d0b7 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -1015,12 +1015,11 @@ __bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) if (flags == BPF_TASK_ITER_ALL_PROCS) goto get_next_task; - kit->pos = next_thread(kit->pos); - if (kit->pos == kit->task) { - if (flags == BPF_TASK_ITER_PROC_THREADS) { - kit->pos = NULL; + kit->pos = __next_thread(kit->pos); + if (!kit->pos) { + if (flags == BPF_TASK_ITER_PROC_THREADS) return pos; - } + kit->pos = kit->task; } else return pos; From patchwork Tue Nov 14 16:32:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Nesterov X-Patchwork-Id: 13455606 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E771D2E8 for ; Tue, 14 Nov 2023 16:33:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R0hvXgBQ" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FA6613D for ; Tue, 14 Nov 2023 08:33:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1699979633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=u12QSukl/sf9yDwpJ5WMUElypfqqvvHxept5As9gCMA=; b=R0hvXgBQOkxcPSfcg4l7T/zYxIl0FZWryUuC7DXWrEOWloEf1xNCiRiPsq6vR7sP3TcBp3 mSeJA3+GcKexgYYT9kTrb/f+xGYsT7Vhm4pq4dgqaxEWbUaAUK7qrZyV014lwcgxxvoc10 mGkcwaFrRBo4kghl4bK5ECNAQ07pcEI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-528-snAvf6GpMGSW2NMVnMYu4w-1; Tue, 14 Nov 2023 11:33:47 -0500 X-MC-Unique: snAvf6GpMGSW2NMVnMYu4w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3CF7B85C6EA; Tue, 14 Nov 2023 16:33:47 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.225.253]) by smtp.corp.redhat.com (Postfix) with SMTP id 3EC072026D66; Tue, 14 Nov 2023 16:33:45 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Tue, 14 Nov 2023 17:32:42 +0100 (CET) Date: Tue, 14 Nov 2023 17:32:39 +0100 From: Oleg Nesterov To: Alexei Starovoitov , Yonghong Song Cc: Chuyi Zhou , Daniel Borkmann , Kui-Feng Lee , linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH 3/3] bpf: bpf_iter_task_next: use next_task(kit->task) rather than next_task(kit->pos) Message-ID: <20231114163239.GA903@redhat.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231114163211.GA874@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Patchwork-Delegate: bpf@iogearbox.net This looks more clear and simplifies the code. While at it, remove the unnecessary initialization of pos/task at the start of bpf_iter_task_new(). Note that we can even kill kit->task, we can just use pos->group_leader, but I don't understand the BUILD_BUG_ON() checks in bpf_iter_task_new(). Signed-off-by: Oleg Nesterov Acked-by: Yonghong Song --- kernel/bpf/task_iter.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index d42e08d0d0b7..e5c3500443c6 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -978,7 +978,6 @@ __bpf_kfunc int bpf_iter_task_new(struct bpf_iter_task *it, BUILD_BUG_ON(__alignof__(struct bpf_iter_task_kern) != __alignof__(struct bpf_iter_task)); - kit->task = kit->pos = NULL; switch (flags) { case BPF_TASK_ITER_ALL_THREADS: case BPF_TASK_ITER_ALL_PROCS: @@ -1016,18 +1015,15 @@ __bpf_kfunc struct task_struct *bpf_iter_task_next(struct bpf_iter_task *it) goto get_next_task; kit->pos = __next_thread(kit->pos); - if (!kit->pos) { - if (flags == BPF_TASK_ITER_PROC_THREADS) - return pos; - kit->pos = kit->task; - } else + if (kit->pos || flags == BPF_TASK_ITER_PROC_THREADS) return pos; get_next_task: - kit->pos = next_task(kit->pos); - kit->task = kit->pos; - if (kit->pos == &init_task) + kit->task = next_task(kit->task); + if (kit->task == &init_task) kit->pos = NULL; + else + kit->pos = kit->task; return pos; }