From patchwork Tue Jul 9 20:42:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99E6DC3DA45 for ; Tue, 9 Jul 2024 20:42:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B1EC6B0098; Tue, 9 Jul 2024 16:42:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23A276B0099; Tue, 9 Jul 2024 16:42:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B4576B009A; Tue, 9 Jul 2024 16:42:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DDACA6B0098 for ; Tue, 9 Jul 2024 16:42:58 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 84DC3161A64 for ; Tue, 9 Jul 2024 20:42:58 +0000 (UTC) X-FDA: 82321388436.24.D01D4F6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id CB9DA180012 for ; Tue, 9 Jul 2024 20:42:56 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oo9z9gwt; spf=pass (imf06.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557752; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=92WXwhF8uyyedF2P8wMs+S83rNVQ5kpF41y/v3AKXCg=; b=kBXdKsItIW27S98lteGihs8lIJ+qabBDB6J5Jg1fksUkIj9DmRjNsgHqnp0frZ5v364lgn 82BtfjXOaOL4L10xziSz03xP55CVKkennqN+UZUFtLlClkW7uUSB76kVCInjDWyw8ooIOB I+0JENS4mtEQzlBGqYRqVF7eFMEop4s= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oo9z9gwt; spf=pass (imf06.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557752; a=rsa-sha256; cv=none; b=Lek1p/Imwb1Jiy3J164InfsuVaxbJmY0pLm5TDdG7+Puwy7XPpib8sowMtq6QlkB4RsOYZ VdGI8qGcgZAs0a7+mSGp6wl3h/0zD5TNtFnAsKqixLwvl6tBWtU+BVaIDZhyrtAf69Vxoz Xf+fnG8CKEFYBOmHcquKy2/zIW9S8fA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id EEA49616BE; Tue, 9 Jul 2024 20:42:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3EE34C32786; Tue, 9 Jul 2024 20:42:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557775; bh=q2ogziqHOj9Wq6uIoecdUqbK1pcPKH+zumPWi1mLFts=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oo9z9gwthDAyYsZ1OiO1S9qCLIYINYAhAiiO0Mk/7yJFcSrrKdtpBx+1GYwVYjKj4 gLdlURjGIMGovX2dPU8znbE3pD2R1yXIDNbcgMD5lFObRWWpCrEcb7zVPYpeIvx/Oq 6BIdtpcH/Az7ffN9bsI6zSN2mwLb0BbQc9SGXJFAr4gg9ppIC9WChbyq8Mrs0WulK3 c8BroOjnC4XsVoHSZMvfmgkixO94QXWGJpk2B4onTlmk86rqb9ThGVwwh9f4052yjc zR90Za14qx54PrEh4WuQTXvImXR9Z5x80VPQ3PT29Wn4n3XZlerSbyaAYDw0b7tvmZ 4DmFAWegKkG0A== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 01/10] lib/buildid: add single page-based file reader abstraction Date: Tue, 9 Jul 2024 13:42:36 -0700 Message-ID: <20240709204245.3847811-2-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 3m1qus6b6tjbpiu86uzf7ir3est5kj4i X-Rspam-User: X-Rspamd-Queue-Id: CB9DA180012 X-Rspamd-Server: rspam02 X-HE-Tag: 1720557776-93872 X-HE-Meta: U2FsdGVkX1/N37Dgqom64HE5jU31XXZ7VLB3JlnW6M9UJchHtBfd2NdkVtUBevZn6MmRZYGkr8xKM+q3D4dQDObK9gwrwHcXLy3GapXWdqoo6tuHKi1g7myYbhT1RcZ+ROkA4z4xAddHelLlunMbkLOxq/djbYcOsMRBjfTNegR2dfvtOZr7MEYcgqSwjM3fI4gaN/F/un/2WTpAFBNNpxqcDYru6MOYyhN0bV6YW/wFouQnfhl3qTM1EW+t/vNxQa6+0rdNeuXqxV8yNmtpGwJubtzDpW/Yjh4Fw4mfS7UsHlB4llFEwhBWazJzj33okB7mNm7fAeGQzMCVGmr2tPV5punpgFeVEKUIr0ZrTbCnKnW3GhRZ19B+Xy/HLewJb1HWrmPlpwYrfcETEKE4NRrVpQohkxynPsjru09mnELbo4dOK2t1VbBHiRbm8Ozarw+vril7aHlaolJpJHhAtE7La5CS9BBEurfgPZJjMjB/obSk6ttZff58GlMVtf5HTlkr1mqdXFcIqANkCe5YII266RNiBu30QEVriKSaGFNdPFiocZq/bzMZsrMsCfnOFR9YYpzhMfOSPOepknxQ7UNV3sy9Rj31OHGd4KtUDUaY1meZaaSaNLojUP/3As/C68xM4hKigjI7FqoukUejNUnJy1S4qUNQElZPQ7MwXao8dKSuRFkCCngbImGtPn+7k752wjSAeby0tvJ4wAyZJJuIy4P9WdibZk6u/E6rBqcJY1j8bh/vx7SA/0UFmx0xC6o1p81wiGt/b2LxZWLbtEEJ/Hr80GlAAh12xsQ70XgXqpBfi7yrMRv+6RnM86VDO8JD1S5kVuBBLn9jTeXgp2NMWVGJggVqdyGpQvlSUTPylSXQq1qyQkWVK0aF6PTxYdrCSVjIbBJ8o25YqR7efLvklSFXJ6DrhGO55/AzWIupyI6Ula4HUysUsUtB+GSJWK6OxI68yJ1VgTyLijZ hIk+LXf3 OUqOPHdlQjpfW6SLncsupLaZeuKxtGzxb8BNwSmROC2FQZ9E7Cs6dasUr2+dRmOSCtbPJlnBR88sqN75p0e9nbVcySoHCVejLITCA1PtkO+yWPDvTBse/ME6u27hpp4qnJq6rSc83vYsDWNFjtArMWUSyqQD+VmckeP++MfSv35Qv53QNYpKs0wRecaMsWy07cecM9ybApiDnsWG39n8vWt3IEDE68IfsWtRfX2o43JC8BHU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add freader abstraction that transparently manages fetching and local mapping of the underlying file page(s) and provides a simple direct data access interface. freader_fetch() is the only and single interface necessary. It accepts file offset and desired number of bytes that should be accessed, and will return a kernel mapped pointer that caller can use to dereference data up to requested size. Requested size can't be bigger than the size of the extra buffer provided during initialization (because, worst case, all requested data has to be copied into it, so it's better to flag wrongly sized buffer unconditionally, regardless if requested data range is crossing page boundaries or not). If page is not paged in, or some of the conditions are not satisfied, NULL is returned and more detailed error code can be accessed through freader->err field. This approach makes the usage of freader_fetch() cleaner. To accommodate accessing file data that crosses page boundaries, user has to provide an extra buffer that will be used to make a local copy, if necessary. This is done to maintain a simple linear pointer data access interface. We switch existing build ID parsing logic to it, without changing or lifting any of the existing constraints, yet. This will be done separately. Given existing code was written with the assumption that it's always working with a single (first) page of the underlying ELF file, logic passes direct pointers around, which doesn't really work well with freader approach and would be limiting when removing the single page limitation. So we adjust all the logic to work in terms of file offsets. There is also a memory buffer-based version (freader_init_from_mem()) for cases when desired data is already available in kernel memory. This is used for parsing vmlinux's own build ID note. In this mode assumption is that provided data starts at "file offset" zero, which works great when parsing ELF notes sections, as all the parsing logic is relative to note section's start. Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 278 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 217 insertions(+), 61 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 7954dd92e36c..1442a2483a8b 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -8,38 +8,174 @@ #define BUILD_ID 3 +struct freader { + void *buf; + u32 buf_sz; + int err; + union { + struct { + struct address_space *mapping; + struct page *page; + void *page_addr; + u64 file_off; + }; + struct { + const char *data; + u64 data_sz; + }; + }; +}; + +static void freader_init_from_file(struct freader *r, void *buf, u32 buf_sz, + struct address_space *mapping) +{ + memset(r, 0, sizeof(*r)); + r->buf = buf; + r->buf_sz = buf_sz; + r->mapping = mapping; +} + +static void freader_init_from_mem(struct freader *r, const char *data, u64 data_sz) +{ + memset(r, 0, sizeof(*r)); + r->data = data; + r->data_sz = data_sz; +} + +static void freader_put_page(struct freader *r) +{ + if (!r->page) + return; + kunmap_local(r->page_addr); + put_page(r->page); + r->page = NULL; +} + +static int freader_get_page(struct freader *r, u64 file_off) +{ + pgoff_t pg_off = file_off >> PAGE_SHIFT; + + freader_put_page(r); + + r->page = find_get_page(r->mapping, pg_off); + if (!r->page) + return -EFAULT; /* page not mapped */ + + r->page_addr = kmap_local_page(r->page); + r->file_off = file_off & PAGE_MASK; + + return 0; +} + +static const void *freader_fetch(struct freader *r, u64 file_off, size_t sz) +{ + int err; + + /* provided internal temporary buffer should be sized correctly */ + if (WARN_ON(r->buf && sz > r->buf_sz)) { + r->err = -E2BIG; + return NULL; + } + + if (unlikely(file_off + sz < file_off)) { + r->err = -EOVERFLOW; + return NULL; + } + + /* working with memory buffer is much more straightforward */ + if (!r->buf) { + if (file_off + sz > r->data_sz) { + r->err = -ERANGE; + return NULL; + } + return r->data + file_off; + } + + /* check if we need to fetch a different page first */ + if (!r->page || file_off < r->file_off || file_off >= r->file_off + PAGE_SIZE) { + err = freader_get_page(r, file_off); + if (err) { + r->err = err; + return NULL; + } + } + + /* if requested data is crossing page boundaries, we have to copy + * everything into our local buffer to keep a simple linear memory + * access interface + */ + if (file_off + sz > r->file_off + PAGE_SIZE) { + int part_sz = r->file_off + PAGE_SIZE - file_off; + + /* copy the part that resides in the current page */ + memcpy(r->buf, r->page_addr + (file_off - r->file_off), part_sz); + + /* fetch next page */ + err = freader_get_page(r, r->file_off + PAGE_SIZE); + if (err) { + r->err = err; + return NULL; + } + + /* copy the rest of requested data */ + memcpy(r->buf + part_sz, r->page_addr, sz - part_sz); + + return r->buf; + } + + /* if data fits in a single page, just return direct pointer */ + return r->page_addr + (file_off - r->file_off); +} + +static void freader_cleanup(struct freader *r) +{ + freader_put_page(r); +} + /* * Parse build id from the note segment. This logic can be shared between * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are * identical. */ -static int parse_build_id_buf(unsigned char *build_id, - __u32 *size, - const void *note_start, - Elf32_Word note_size) +static int parse_build_id_buf(struct freader *r, + unsigned char *build_id, __u32 *size, + u64 note_offs, Elf32_Word note_size) { - Elf32_Word note_offs = 0, new_offs; + const char note_name[] = "GNU"; + const size_t note_name_sz = sizeof(note_name); + u64 build_id_off, new_offs, note_end = note_offs + note_size; + u32 build_id_sz; + const Elf32_Nhdr *nhdr; + const char *data; - while (note_offs + sizeof(Elf32_Nhdr) < note_size) { - Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_offs); + while (note_offs + sizeof(Elf32_Nhdr) < note_end) { + nhdr = freader_fetch(r, note_offs, sizeof(Elf32_Nhdr) + note_name_sz); + if (!nhdr) + return r->err; if (nhdr->n_type == BUILD_ID && - nhdr->n_namesz == sizeof("GNU") && - !strcmp((char *)(nhdr + 1), "GNU") && + nhdr->n_namesz == note_name_sz && + !strcmp((char *)(nhdr + 1), note_name) && nhdr->n_descsz > 0 && nhdr->n_descsz <= BUILD_ID_SIZE_MAX) { - memcpy(build_id, - note_start + note_offs + - ALIGN(sizeof("GNU"), 4) + sizeof(Elf32_Nhdr), - nhdr->n_descsz); - memset(build_id + nhdr->n_descsz, 0, - BUILD_ID_SIZE_MAX - nhdr->n_descsz); + + build_id_off = note_offs + sizeof(Elf32_Nhdr) + ALIGN(note_name_sz, 4); + build_id_sz = nhdr->n_descsz; + + /* freader_fetch() will invalidate nhdr pointer */ + data = freader_fetch(r, build_id_off, build_id_sz); + if (!data) + return r->err; + + memcpy(build_id, data, build_id_sz); + memset(build_id + build_id_sz, 0, BUILD_ID_SIZE_MAX - build_id_sz); if (size) - *size = nhdr->n_descsz; + *size = build_id_sz; return 0; } + new_offs = note_offs + sizeof(Elf32_Nhdr) + - ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4); + ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4); if (new_offs <= note_offs) /* overflow */ break; note_offs = new_offs; @@ -48,73 +184,87 @@ static int parse_build_id_buf(unsigned char *build_id, return -EINVAL; } -static inline int parse_build_id(const void *page_addr, +static inline int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *size, - const void *note_start, + u64 note_start_off, Elf32_Word note_size) { /* check for overflow */ - if (note_start < page_addr || note_start + note_size < note_start) + if (note_start_off + note_size < note_start_off) return -EINVAL; /* only supports note that fits in the first page */ - if (note_start + note_size > page_addr + PAGE_SIZE) + if (note_start_off + note_size > PAGE_SIZE) return -EINVAL; - return parse_build_id_buf(build_id, size, note_start, note_size); + return parse_build_id_buf(r, build_id, size, note_start_off, note_size); } /* Parse build ID from 32-bit ELF */ -static int get_build_id_32(const void *page_addr, unsigned char *build_id, - __u32 *size) +static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *size) { - Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr; - Elf32_Phdr *phdr; - int i; + const Elf32_Ehdr *ehdr; + const Elf32_Phdr *phdr; + __u32 phnum, i; + + ehdr = freader_fetch(r, 0, sizeof(Elf32_Ehdr)); + if (!ehdr) + return r->err; + + /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ + phnum = ehdr->e_phnum; /* only supports phdr that fits in one page */ - if (ehdr->e_phnum > - (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) + if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) return -EINVAL; - phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr)); + for (i = 0; i < phnum; ++i) { + phdr = freader_fetch(r, i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); + if (!phdr) + return r->err; - for (i = 0; i < ehdr->e_phnum; ++i) { - if (phdr[i].p_type == PT_NOTE && - !parse_build_id(page_addr, build_id, size, - page_addr + phdr[i].p_offset, - phdr[i].p_filesz)) + if (phdr->p_type == PT_NOTE && + !parse_build_id(r, build_id, size, phdr->p_offset, phdr->p_filesz)) return 0; } return -EINVAL; } /* Parse build ID from 64-bit ELF */ -static int get_build_id_64(const void *page_addr, unsigned char *build_id, - __u32 *size) +static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *size) { - Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr; - Elf64_Phdr *phdr; - int i; + const Elf64_Ehdr *ehdr; + const Elf64_Phdr *phdr; + __u32 phnum, i; + + ehdr = freader_fetch(r, 0, sizeof(Elf64_Ehdr)); + if (!ehdr) + return r->err; + + /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ + phnum = ehdr->e_phnum; /* only supports phdr that fits in one page */ - if (ehdr->e_phnum > - (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) + if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) return -EINVAL; - phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr)); + for (i = 0; i < phnum; ++i) { + phdr = freader_fetch(r, i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); + if (!phdr) + return r->err; - for (i = 0; i < ehdr->e_phnum; ++i) { - if (phdr[i].p_type == PT_NOTE && - !parse_build_id(page_addr, build_id, size, - page_addr + phdr[i].p_offset, - phdr[i].p_filesz)) + if (phdr->p_type == PT_NOTE && + !parse_build_id(r, build_id, size, phdr->p_offset, phdr->p_filesz)) return 0; } + return -EINVAL; } +/* enough for Elf64_Ehdr, Elf64_Phdr, and all the smaller requests */ +#define MAX_FREADER_BUF_SZ 64 + /* * Parse build ID of ELF file mapped to vma * @vma: vma object @@ -126,22 +276,25 @@ static int get_build_id_64(const void *page_addr, unsigned char *build_id, int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { - Elf32_Ehdr *ehdr; - struct page *page; - void *page_addr; + const Elf32_Ehdr *ehdr; + struct freader r; + char buf[MAX_FREADER_BUF_SZ]; int ret; /* only works for page backed storage */ if (!vma->vm_file) return -EINVAL; - page = find_get_page(vma->vm_file->f_mapping, 0); - if (!page) - return -EFAULT; /* page not mapped */ + freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file->f_mapping); + + /* fetch first 18 bytes of ELF header for checks */ + ehdr = freader_fetch(&r, 0, offsetofend(Elf32_Ehdr, e_type)); + if (!ehdr) { + ret = r.err; + goto out; + } ret = -EINVAL; - page_addr = kmap_local_page(page); - ehdr = (Elf32_Ehdr *)page_addr; /* compare magic x7f "ELF" */ if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG) != 0) @@ -152,12 +305,11 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, goto out; if (ehdr->e_ident[EI_CLASS] == ELFCLASS32) - ret = get_build_id_32(page_addr, build_id, size); + ret = get_build_id_32(&r, build_id, size); else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64) - ret = get_build_id_64(page_addr, build_id, size); + ret = get_build_id_64(&r, build_id, size); out: - kunmap_local(page_addr); - put_page(page); + freader_cleanup(&r); return ret; } @@ -171,7 +323,11 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, */ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) { - return parse_build_id_buf(build_id, NULL, buf, buf_size); + struct freader r; + + freader_init_from_mem(&r, buf, buf_size); + + return parse_build_id_buf(&r, build_id, NULL, 0, buf_size); } #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID) || IS_ENABLED(CONFIG_VMCORE_INFO) From patchwork Tue Jul 9 20:42:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 636FAC2BD09 for ; Tue, 9 Jul 2024 20:43:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9CA96B0099; Tue, 9 Jul 2024 16:43:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E49956B009A; Tue, 9 Jul 2024 16:43:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC3C66B009B; Tue, 9 Jul 2024 16:43:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AFE286B0099 for ; Tue, 9 Jul 2024 16:43:01 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 75CA781A17 for ; Tue, 9 Jul 2024 20:43:01 +0000 (UTC) X-FDA: 82321388562.19.91F6333 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id D9B60140003 for ; Tue, 9 Jul 2024 20:42:59 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uymoLYG0; spf=pass (imf26.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WOAQQVIDcPMn/JrdEI1DEueVlKQqj+vq02sOlQag+xE=; b=s7QxKIdmrCoFQH4gxZ8C8e99So9xUogCWGrfzSLx9CEsSYXKjHJ570qCoOLPQep9pqP9Uf PrK6NtgAUEA1te9rAKEBaIC4/vqMRjWWMf8gNrqltZaTcYzqR+QkpFwtfwx0tkYC34FfYi l/jK9T9aWo6sxpUNBK68fSWIq3fQrrw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=uymoLYG0; spf=pass (imf26.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557765; a=rsa-sha256; cv=none; b=DUl8wmnFxvkQYp+hJ2vFLwYBf3vSvA3gm9HCXLtsfqFV/uYZKy3jCKaVmVG9U14lYhMNwE GO5JtTYt7ppu8ONZGcm8Il+07fR4y5s/fqbxc4HNpK8v6b7OOEf7zAw2hN5du8DHLf/Jn2 6+XZifbBi0nH3KoAOLw+uXZfJoWVuM8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0DD9F616AC; Tue, 9 Jul 2024 20:42:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F973C32786; Tue, 9 Jul 2024 20:42:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557778; bh=C6pZlI4b95ZXtBjMHdytNmZo2a2DbuYqAFHnzJFYuwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uymoLYG0Q9SwCI7KQesqLBFvnArB0/gIUKz3koEE36GLmnH/t6Hnm7YORfXF3TtLR UKV0yis/gKz6Xvzr4AmQRHAOOAl9ZY30qvaanFYSSOXcssDkZtWcighSgsEbkol3PC tbPi/R15irjarwVMdKzVktzp/HPknwURi5Xgo13swWH1SYajub6Hyuocw+P4Lyl7da 77OH9LrPv5JINgUMj1vVlgU0Pcdvi2WJ2ntIj8pH3+wSHEE44KzlMy/r8bAdDWXV+p BRBfzJs3zH6o0SiqpXQ+4qiMCv7zqV0iILQhHpdlRKyh8fVHhjq0M20WNBvrofiX+4 ucJ88atYNDVeg== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 02/10] lib/buildid: take into account e_phoff when fetching program headers Date: Tue, 9 Jul 2024 13:42:37 -0700 Message-ID: <20240709204245.3847811-3-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D9B60140003 X-Stat-Signature: hko9sdqxrdqoa6zou3zbzq6sqtowdraz X-HE-Tag: 1720557779-316391 X-HE-Meta: U2FsdGVkX181heO2QZ6j/XyI2CHTyAUvSqWE63ivj/K52cUL2hkrAFShLgA5NR/rJFVXcmqCIQl0MQatRBLx7uPpmEyA8hNX3HMCxzOQUevz6eoOQER6DqyaFrLQtfTxcFuZkjO0Ip3go1TBC/vb1FoxPnq38G4itmW5/qBMKnY1O6WvrtQfWxvKbE5U3ity7yatS2qOOYZ3SgJ8RjrlGR8nXN3GXhM2prG9glHF4ou1iR7oQDGOlBA9WWK5F6CCG1r1lH0KSr/cFPUYqBd7CEhf5MeUlV07Kkj34AnOKTkQN+uhq4tJjJvdWHQSVJFFHrmJop0xGekcDauflpgrkQRLzRqKqR1mJjjC2jvILYvRN00G50uQYNmABslbkHatGhAfwsqoRKcfSeIXwiRNmcejx0D3vUpT+sSKx+2O3vxZD5boOAM//IiHNn08rX9q2VWYnxQdRoy0CswlPFvY6f8KmOhJ9ufYrfir+k/NjiUYv+UYYaKpDvzB8kOnYO9cfIFKHHYafwlaetL8VUQg62vfKbm7brEd1FSPidsiQg3dWH3hE0JiKvpHszQyltQQdbo2IJtHWSRHoJxXRhDbV/4Oe47aNXnsQRz/E0QkjSQOJrrsvVhvBnvHjADNUZedI3wxJ15iSh3mzkjpFzIyLIK6ZS+eVccE2kCtj0AJ0+CxgmNuq7mZ1NxO2zEnJzj1NEY5AsStjLmIEDubWqtQvpzOXHSzxt9XkIA/QAGMrEY8vb8DazEenXRAdPsL6fQJiTcAemx2T2f/zOEHPwCY3IVAm33XhSOwZc0dv9cyP68VDTNC+XOF81ih0h/RSeOFXeMgc/3nbmjL33J2UxR4VGYmBCtaBbclNz4RVFwMENzAXS9FVJxgv52x/RUd7zKs2d06TIFlUVAkLW06zdgJS8lfGNgEXxi9Rg5jxx0qfsh/DgtosrUQAriBhFZb6fdQQDsT2QrGVlBULjp9VTo 4rWWT3Ua sQ/9PGkP+ZNdq04KXZUSbBwQEUZGgPUT3kLe5gzNPwd1B2HT0hw8NqBWf5MeGGw1JrfWTq37p4HNT/aIAqfPJKYpcluHZcRNjlDeOH22JpM+/K8DLi266J40aeQ6UTtEiPc25kI/9p0DL72IZA8Jsh+RMkfZ8lDuyNBmGwtYXqkSnYrizi/fCCykGm8q3DH61UJk9m6B3wKYLIdTnrO8+1tDGtE2yZm/oyXPMr4TsU6YFHSp8UMFSCx449GHrD9CVq+9LrpktxS39KbFJ3phSQXf8H3DfceGhg+OQmspfNIpP7Io3uu5CW79ZlQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Current code assumption is that program (segment) headers are following ELF header immediately. This is a common case, but is not guaranteed. So take into account e_phoff field of the ELF header when accessing program headers. Reported-by: Alexey Dobriyan Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 1442a2483a8b..ce48ffab4111 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -206,7 +206,7 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si { const Elf32_Ehdr *ehdr; const Elf32_Phdr *phdr; - __u32 phnum, i; + __u32 phnum, phoff, i; ehdr = freader_fetch(r, 0, sizeof(Elf32_Ehdr)); if (!ehdr) @@ -214,13 +214,14 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = ehdr->e_phnum; + phoff = READ_ONCE(ehdr->e_phoff); /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) return -EINVAL; for (i = 0; i < phnum; ++i) { - phdr = freader_fetch(r, i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); + phdr = freader_fetch(r, phoff + i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); if (!phdr) return r->err; @@ -237,6 +238,7 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si const Elf64_Ehdr *ehdr; const Elf64_Phdr *phdr; __u32 phnum, i; + __u64 phoff; ehdr = freader_fetch(r, 0, sizeof(Elf64_Ehdr)); if (!ehdr) @@ -244,13 +246,14 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ phnum = ehdr->e_phnum; + phoff = READ_ONCE(ehdr->e_phoff); /* only supports phdr that fits in one page */ if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) return -EINVAL; for (i = 0; i < phnum; ++i) { - phdr = freader_fetch(r, i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); + phdr = freader_fetch(r, phoff + i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); if (!phdr) return r->err; From patchwork Tue Jul 9 20:42:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9723C2BD09 for ; Tue, 9 Jul 2024 20:43:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 575016B009B; Tue, 9 Jul 2024 16:43:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 522B66B009C; Tue, 9 Jul 2024 16:43:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C3086B009D; Tue, 9 Jul 2024 16:43:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1630D6B009B for ; Tue, 9 Jul 2024 16:43:05 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C1E15121A51 for ; Tue, 9 Jul 2024 20:43:04 +0000 (UTC) X-FDA: 82321388688.25.F95D833 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 2A77912000D for ; Tue, 9 Jul 2024 20:43:02 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UOtP9RNv; spf=pass (imf29.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557767; a=rsa-sha256; cv=none; b=CSQpqzY29rTcWZjAXrVD09c8koiMZ+KqDs2+m5bkm707o1fIUbeLd9KhVm+O4sXRKI/zhd JrM2SMJnlOD/cULf5ENPkMpLvwevQpANzjfDQQetieohfgYqxg68twAxMFHOwShgQXmfIa HLNGCVkC3/+fMpTFP83NKLlqM0g/yX4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UOtP9RNv; spf=pass (imf29.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557767; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=30l0Nzy5VjI9ZJ/8KJLNGnELHLJCo9GumCm2dYcbpXc=; b=SjQTwZdhmUFKek0WfRSS0Gyg19KBjXt4tYckWW2A4/I0M9is2QOqCAVadqyVxPY1K5//Kw jHl9QQzinrJHu/q7VbZhV6rh7KtqoXGhIxI8Qj0sYvgx4PLFxiN4XPE1POoBcVDfehaxD7 BXGn1gJuRLzLG0Ueif+rwOSYtHP1Uxs= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 4C8B5616BE; Tue, 9 Jul 2024 20:43:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D758EC4AF07; Tue, 9 Jul 2024 20:43:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557782; bh=azpAbr+6SOKGRgY4Eh1WfByGTMRsHGb35RAVJRMiCV8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UOtP9RNv/JJu5/jCkyS+M9BS7SUxW2ejks9KZ2ihPkFrlS7aAoZk8sxAThaQUKtKe FGTCLUADHeEpuzCEU2WvXjl3bJEeV/6zSrDs3zDGSWTxqoTAHEGKhdKq/h6MEpSHJd 4RcAwWx+IpDrpEJcMitOcH3zc5WznjT4ZXes4BpdcPH2uN/Y7BYcfTRwEQ6buTU2RF iejKag7U/HcGzFOXmA5TICySYvNi/wKo9oM21D58l4OipS1nccsKnT3V257DXvZ9LG hMrwJwzDYuzvt/BPe7NwBXaBKtjGPY3ujolo3mSwaRmdetENQvxL3m4XChvN0UVtzM D94czy+e+RVyQ== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 03/10] lib/buildid: remove single-page limit for PHDR search Date: Tue, 9 Jul 2024 13:42:38 -0700 Message-ID: <20240709204245.3847811-4-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: aotk5hmpbco1iqqqu813idq5z1qkjyok X-Rspamd-Queue-Id: 2A77912000D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1720557782-208233 X-HE-Meta: U2FsdGVkX1+RbFFC/yeA4eV17AcKNO1CJJNvSnPLkkAy0yTG00dD2fA6Py1DxWRyu79vxxFK3Djo/FZb6uolFDwx4DTvhVuoMScjM5/YF3i5hhK79fCglA1pWTC1EZrYgLcQT199esoXNfclrOSnXkBTCoCpsq86LGkfYEAQ4HbeUc86Vs1Xc5y6jy50SC9aOqgnM9R999UIL66kCwTj+LCwmErauwuqV0tpC+RmHXrZmRhmz9Xa6/NHvQ3v6d2Q9OR0b0kdek6jy7/K7ZSwSMJME324sR6231JrEbB18yXPGCKgUwLKiEdv7LSK34VQrZNWGJ542N837g9i9lNnxlWk6/06lVRogCfe9Jq4sTdsiEvm3n1ZTXY4bH5vOFthlT8AOuXyEQapVi871E+5mLThPi8CxoXPRecpLBw1U/+jht9MGUhFlTiHNNPxINBK89M98nzAq10DPhdcNyGMeha+Pg4dmnluf2kRI/77+cKRr9tGerbPrX86ffPef84gEcPOMCQUE5G15Vx0wkh5tJth0oX4TJHvY0RJo7njzMyIcJ4G+H7pEryZHd8g1T5+n7od6yokjI+ukkaaep6Q1cHOw8rR5rTdxo9uHpWRjxyfUcVx3epPLQtUgfumuOa7BPhpzl3i2LM/0ujVZqhRS0WJ2+iBoKSunoZTwmiYujuWTvnZ/FgKBOr27QQOKEUZ+Df6G3qKiYdWXSM0eRtrGmNSGDiB152PlhpIHzQFK8bHsvYBDSlaPi1JDEd9xotzBdbxiZYcLtkntA/NmDkYmN+y5tdd+liR409NQ97tfxtzpgBBXUWwdlKoFtKCKJ4x18rnD+Jle0tL7cfK3iXecpZiwvJ2kIOlS2XWrDITyfrajXpRPTt+AwuGgbHR1IDbJ/SunfY+ZemHn++7kLOMOdldrKk6BCkx0SNvDK9VZ0antJOpMRm4Si9J4kgHQsZ4L+k49NV6bnOVkULIPJF L/Tcupn/ Z5nWEedZV4mjiqcfuqtyjSFtNKcvklCSp2AxCjjMoUvLEfg+rfMrnWWuipOq44ucTye3750/fvj4UfgoCNfIPzpaJVKKOcHgk664fPv39E6EOpcWfBXEpLwwdL1MARD0/w0qajy250rMzFXcTGRX9wXjpvGLsbUVGIOV4eRYJJnGae0mKp0MjvYFgTlDAzyKIOJzy+naXx6F93S+3HZdR2VIBY8d2Ds/4YPytm4ei0WZAacY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that freader allows to access multiple pages transparently, there is no need to limit program headers to the very first ELF file page. Remove this limitation, but still put some sane limit on amount of program headers that we are willing to iterate over (set arbitrarily to 256). Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index ce48ffab4111..49fcb9a549bf 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -8,6 +8,8 @@ #define BUILD_ID 3 +#define MAX_PHDR_CNT 256 + struct freader { void *buf; u32 buf_sz; @@ -216,9 +218,9 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si phnum = ehdr->e_phnum; phoff = READ_ONCE(ehdr->e_phoff); - /* only supports phdr that fits in one page */ - if (phnum > (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr)) - return -EINVAL; + /* set upper bound on amount of segments (phdrs) we iterate */ + if (phnum > MAX_PHDR_CNT) + phnum = MAX_PHDR_CNT; for (i = 0; i < phnum; ++i) { phdr = freader_fetch(r, phoff + i * sizeof(Elf32_Phdr), sizeof(Elf32_Phdr)); @@ -248,9 +250,9 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si phnum = ehdr->e_phnum; phoff = READ_ONCE(ehdr->e_phoff); - /* only supports phdr that fits in one page */ - if (phnum > (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr)) - return -EINVAL; + /* set upper bound on amount of segments (phdrs) we iterate */ + if (phnum > MAX_PHDR_CNT) + phnum = MAX_PHDR_CNT; for (i = 0; i < phnum; ++i) { phdr = freader_fetch(r, phoff + i * sizeof(Elf64_Phdr), sizeof(Elf64_Phdr)); From patchwork Tue Jul 9 20:42:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63B85C2BD09 for ; Tue, 9 Jul 2024 20:43:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DFD0E6B009D; Tue, 9 Jul 2024 16:43:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DACB66B009E; Tue, 9 Jul 2024 16:43:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C262B6B009F; Tue, 9 Jul 2024 16:43:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A0DF36B009D for ; Tue, 9 Jul 2024 16:43:08 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1F40E413DF for ; Tue, 9 Jul 2024 20:43:08 +0000 (UTC) X-FDA: 82321388856.15.45695E2 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 40F7114001F for ; Tue, 9 Jul 2024 20:43:06 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WzRoCXxx; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557755; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CxajWYdjrgD7wontsDXsiAtgcbkzL0TRv4p1vUoxG0o=; b=JZm0xcxB7LXb3OBbhPfRCLfry2NyjgJYjSbD7Ts/lBpHhZTXBtfRR626Ig2LeLiAnjROZ0 Uu7Orx+me1jhumTj7ejP5aCfCVyFgSYruqlL//NunkQjqoTBfCxg4OnAoUNlSSre124dMC 6YC4nbNxqlN1n2MtC+jh18rrw3gSLr8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557755; a=rsa-sha256; cv=none; b=vewhFLyRHIj00c7wFKbfgflNf7e+9bIAZIqQUJnxUG4tJRz4bChNpw2lJQrT1/0CCA9WGd 94CNXnRpNgAP0hoDb/bnl4OHIT5kZBAD774l14ESjGgOjQ1uIv35EhKJQtLKR5p9AiXHHU 5qRn0l0U0tQNjL3jzXTycfp9DQX4Xn4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WzRoCXxx; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 781946166E; Tue, 9 Jul 2024 20:43:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0CF29C3277B; Tue, 9 Jul 2024 20:43:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557785; bh=O2hhW1KJHjeOM5cr1mZFJ2v06umN6X8sTn4Mje/nOWo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WzRoCXxxIgU97mgevxSLXRSjJt9eMODm+Xww5DbcnEGGQb32oZygpM+kn7rhk4036 WuHSWR525O8aBUt5+VMpo3Ni89DmSrXi6fKqVw7uh0w6q7fp3VlwcDIF1mB+UX5UeW 0bdX/feM9uiT/gwRPw58Qb7ZWDQ7pRNzMnPN5BLuk7yq+XedvT4gLFi9XYr880NrBP 5mBRcVGdTORwmTEVA2RaWrs7kU6BFQzbzzCjtipMDM3Tb1gKz+99bKMX88AwrPtPgZ bkz73VsSUIbG6us1H1nhwB6z4u1q+yUYcbQ1OIurD950oqwmcFK5gepPRV/og6CODx uBHAedRYltlIA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 04/10] lib/buildid: rename build_id_parse() into build_id_parse_nofault() Date: Tue, 9 Jul 2024 13:42:39 -0700 Message-ID: <20240709204245.3847811-5-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 40F7114001F X-Stat-Signature: 9imkkjhec7cee14t95a7grq7jir1uht8 X-HE-Tag: 1720557786-522672 X-HE-Meta: U2FsdGVkX19JeEYhpXSyHP+janoYB2LKYK1eupp7afR92zVRzJP+2/OxRSOOeulH4cxKRozDoXeGn3PjzVNioGWhyhlARwslglFH/8E1e1qfTtVWukpE7jLn4YS03fcqoonUH4gfaemxSU3a+EifwgqkApslC35Z2itlqoG7gOZvhfRfRwV7m+GffN0iwoxK0lKwtMOUZDJSVmErjjdRFULCFRjToG8cmgQIYlbjRTIa+9xz4P7qqk4LgLIpZ7sfBKhPrzhCdt/xHJyLvu15pbU7Tg2KFVyIKAZw6v0JwSqeymOS4bq7COltU3b2KXHBB8/3QT9SDH6mvI960K2aceQvxVsPdK3+XpCRQqZwJibmGGmdbJnsrIhaCSH9xjEsHe7rfkkEDsnSOkJqgmdQ0UYMEEmGJI1lj0DehmMdyzPsUBn5AWXYEz2NX10m6xVTEkxo1WtPkpWysfOKIuK1gQ6OQSuuH6IrvCXRUDUlGy04IhQ/MG1MFlXQARGhY8PBA5XcSot+qbb+NP5SflhY5JUby/Kif1Ro5qg74ujRFYJEwnEjlx5Gx2ssiRMU8hM2nvM3MY5kK5IaKLvfYAwNmad7Ofto8UTt2ZQTI7BNg4TtjM1DxFXpcBMIxwzGIfH8ub33fLcYCzj7212vHVZIKQQ7pQFmcqSz7tjc8vv5RTngWqKMB8AAn8P1gr+KI+0NERQcFYHTeuyv+eM7ZXDiJDORGAoTmnPNR4U0SxJccmOuI2KnSKwfAhcvAHXn9IcQM0zHAVJp0/OmuYfkAIKI+lIyiQtX92gEKc1LisyYeq/JHp5sX52JmmVsANyJYbjQzyoZ5XvcIZQkmmBgVX47UyZ56EAOFw0nzeQNP8ti6vZcYjORy7v8HmYIeZDKB1eEo9+kKQfFJ9f9/TzFbtGTl4NQBUVmMyz817fQsfWPTBbwdCQDHIe0ySjX4gVheosdBatp9okC+mEU24VR48l 1Co1krTz p28bGYdCKLrx8Fxr2ljxshw3P9kdK1c7PDu/xP2qmvxVtER/hqV4aII76he2/3DGbs4UMpvrptDnZOln2wyLJZ/aq40WfMPrzLXWIS4tG+2DyU2vCeX63rifTATAsE7kwIwil9I+uuE6i64rSLUHA9BY/lE7lr8qxPhPbXwCpRV03/Mmwi9orpLaIQHfH3FGiD3/yIvVt18DasLC8EYiFeNcmpqBuwuusYMFNhmPZVafn1JLC0ArXZ4ToWP++LAzEz9Xgi/jcmRl+N1Cxlg1chImmIwRmt+ube7sh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make it clear that build_id_parse() assumes that it can take no page fault by renaming it and current few users to build_id_parse_nofault(). Also add build_id_parse() stub, which will be implemented in subsequent patches, just to preserve succesful kernel compilation if another upcoming user of build_id_parse() (PROCMAP_QUERY ioctl() for /proc//maps, see [0]) gets merged with bpf-next tree. That ioctl() users of build_id_parse() doesn't have no-page-fault restriction, so it will automatically benefit from sleepable implementation. [0] https://lore.kernel.org/linux-mm/20240627170900.1672542-4-andrii@kernel.org/ Signed-off-by: Andrii Nakryiko --- include/linux/buildid.h | 4 ++-- kernel/bpf/stackmap.c | 2 +- kernel/events/core.c | 2 +- lib/buildid.c | 24 +++++++++++++++++++++--- 4 files changed, 25 insertions(+), 7 deletions(-) diff --git a/include/linux/buildid.h b/include/linux/buildid.h index 20aa3c2d89f7..014a88c41073 100644 --- a/include/linux/buildid.h +++ b/include/linux/buildid.h @@ -7,8 +7,8 @@ #define BUILD_ID_SIZE_MAX 20 struct vm_area_struct; -int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, - __u32 *size); +int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size); +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size); int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size); #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID) || IS_ENABLED(CONFIG_VMCORE_INFO) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index c99f8e5234ac..770ae8e88016 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -156,7 +156,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, goto build_id_valid; } vma = find_vma(current->mm, ips[i]); - if (!vma || build_id_parse(vma, id_offs[i].build_id, NULL)) { + if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; id_offs[i].ip = ips[i]; diff --git a/kernel/events/core.c b/kernel/events/core.c index f0128c5ff278..e3927b688699 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -8812,7 +8812,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event) mmap_event->event_id.header.size = sizeof(mmap_event->event_id) + size; if (atomic_read(&nr_build_id_events)) - build_id_parse(vma, mmap_event->build_id, &mmap_event->build_id_size); + build_id_parse_nofault(vma, mmap_event->build_id, &mmap_event->build_id_size); perf_iterate_sb(perf_event_mmap_output, mmap_event, diff --git a/lib/buildid.c b/lib/buildid.c index 49fcb9a549bf..5f898fee43d7 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -276,10 +276,12 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si * @build_id: buffer to store build id, at least BUILD_ID_SIZE long * @size: returns actual build id size in case of success * - * Return: 0 on success, -EINVAL otherwise + * Assumes no page fault can be taken, so if relevant portions of ELF file are + * not already paged in, fetching of build ID fails. + * + * Return: 0 on success; negative error, otherwise */ -int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, - __u32 *size) +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { const Elf32_Ehdr *ehdr; struct freader r; @@ -318,6 +320,22 @@ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, return ret; } +/* + * Parse build ID of ELF file mapped to VMA + * @vma: vma object + * @build_id: buffer to store build id, at least BUILD_ID_SIZE long + * @size: returns actual build id size in case of success + * + * Assumes faultable context and can cause page faults to bring in file data + * into page cache. + * + * Return: 0 on success; negative error, otherwise + */ +int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +{ + return -EOPNOTSUPP; +} + /** * build_id_parse_buf - Get build ID from a buffer * @buf: ELF note section(s) to parse From patchwork Tue Jul 9 20:42:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64D30C2BD09 for ; Tue, 9 Jul 2024 20:43:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB1326B00A1; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D3A1B6B00A2; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C000F6B00A3; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A30F96B00A1 for ; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 60B5480ADD for ; Tue, 9 Jul 2024 20:43:15 +0000 (UTC) X-FDA: 82321389150.06.29B1177 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf03.hostedemail.com (Postfix) with ESMTP id 1BEC12001C for ; Tue, 9 Jul 2024 20:43:12 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=VKNj5L24; spf=pass (imf03.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557779; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CvVzl7e48xMmAZAYYYfWEbe7EWp8AyF1ZpNYRqbWocc=; b=cedxomsejxYznziyEaY3FB0d66+WRYQY+W69PYo9k84b1Y2lGoYA9iZ2S8utfBrpENe6TU STyp0OgQKsL0E7H4hUTqAFLoa/s6zxZJGooGeVlZKz8oi4t3qMH9yV8o0J8gEJT6GHQe8Z WM3xeRIgZIxYk2tsV4TYIpHfG6cxHr4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=VKNj5L24; spf=pass (imf03.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557779; a=rsa-sha256; cv=none; b=DietnJGw0GTn0tElD52h25fwEc6GmPL3R5CEhYxxgsg7SmTtFxQOe6lKyLz167XIiEEArq rX/Vqn/lF94jvvpVWIWmnas6smlZbv4/iho4gi/HwWoP1B5UUTHq2mbywJT0O1npaZzFiQ 0IDIbIvG8N56lc+DjG7IiW5UO/BeA/A= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 42BB1CE0AEE; Tue, 9 Jul 2024 20:43:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C012C3277B; Tue, 9 Jul 2024 20:43:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557788; bh=+9jpZjOCZAyOPgnFUaMIL0c75rF54bF41tQH1aEgnRg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VKNj5L24TmI7UHUd+8Tc+HQ3syfb1w4WGfckJL3jLjIrwZM6BiJyMio0ayUOCu8ds 5JhEDTMW0GfAj6LRuECJRxoNASWOBsp+9kW4q1Ys4kgoQqAm9Xj+Q32QUjTQTdnV7e vqsYL1E7UnkmTmcS7nZz9Uk7JNRjspulEkktLZrDssyhfoL1q5jY+NMzi7qmCBDwhH BKnFSiC1bhN8KWW9qQSwqL8iUJ7K0NmM7tEC68tzcfhuBjnfs/y7kLJBTCwAcdqWzo shbZBo91h3MwrA/RTUsig7F4hznRtXequGhZ7ElvAw6t7WHlag5ec9Vsj+cKUAH2M2 FTt3Zj/Jcqyig== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko , Omar Sandoval Subject: [PATCH bpf-next 05/10] lib/buildid: implement sleepable build_id_parse() API Date: Tue, 9 Jul 2024 13:42:40 -0700 Message-ID: <20240709204245.3847811-6-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1BEC12001C X-Stat-Signature: ga5c549kchd4e56tr9fx5ou853k6eymz X-HE-Tag: 1720557792-239240 X-HE-Meta: U2FsdGVkX18CoFgNgBnQn9SvkLWDyaneufytQ5AtT8WK+vFLNN5mwC7tsIu0/5UJy5AyWTiq1U+5H9V8zUh2ulOG2HWf+8ehxYfyyZFUKE1x8YH8dLPEyIA548jgs7SLh4dMf3eNGsRzkTasDdntxtfUAp3Wq2Dqqsk+/P+FIU4p16QmOd/kbI9vffIKN4vyvGleJDqdpz7kum6ZjCYpoTHRxR+x3ExriFAQTdgkWhFXYV0toD8seZkK6kYwcfjc6Kt3cbunsaU5tt7QQZ7n/8ViTPva/6hx9KmsA6KQ27e0xgO7qGt1SCYiVQGfvUj4tRlzYwrOn2ciQTF7kOoEd6O9CgcbYOiKJ1GrS+40wsnf/YXzd8mpKE8+bP8qv+HRIUnZOcllRrSky8Bkc3VY/qbYUPpeVi4OkrzGOgfGgNE9Es9VWipIDZRC0Lx4NSXz5/fYFCfH1JUPCyle2vn47E9NsMrZeyNTJQ4ayzSjqkySG9NU+CgmKAokQkC0MohXnJCInOjssv8IFOmg+OXDvgsKbnWcIKsyWO2LGxJ17nNkztSwyH/1oucgRTDqeMvvaSenXZx3AGT0vz12rt3KJL1qYr1xH3A10QiiVmTolbgtndDku3mpjShDe/2atmOC/tAJwphMaqyIqRM3x/OUAfGpFJ4FzPZLcBF673gQDcbB3TL/JrS0YiD8Gc1Yz5MFCVhlyU86JNOdYiGVq7L1W4EVZeE30Ea6jetjvGD11ISMRXM4lqKFAvjjeJboDLYruvofsRe6VGJG9kGqRFXmyxsZWY+k7+v+uBEs6qViv9iFkQyJC/j7IygOoNmKD2kgjhnIE73ghPbnUtthIljgDMdCAdHrtQtGvcO+hqUN6HyDEFG2h8A+s3Qnn4NwgGy+u3LWg6TfJJnh3+3qyOkKmk5+su2wFpo/WNwIRAhKQE3dDha7aN1MHdCkHkF8M6sRmxIrEHav6nGRxKzwSOC YiO7QvLm DW8LaWiXG348TDE7i1ip/SrPCk2P/oIYckgJyuewnlUm6QGjcrZFIDWQnG/tNOYZtk6LOgK85uEBYb4+nxD7h2a4vMSbNGCt1GQd5vjUyBtAmyGWh1T7w9O8my9f8mqk+5ahupLJpECOHpllQFtLDUdBFXGI5z40kW9llvchJx9xFLk71NVbauCSQtjv7Rwep/8PZZS7GDQf/vlQKxkWAqOQcG/V2c/KydAvNq2YfhTnbSdbV94JxKF1LDVwuBTbUksvhjRWzkWzPL5XOZ3UldLCX4CISczpQQLIEl1jFhc/aOezX1+W0agbk9Rm9YgKF6//MG2yUThQk5ka/MO00PwtzVCEam0UyL2SwPeZoE7/m+85WW2j1W52bqCTL+MZ4Yz9izsjmU+kC7nwpyW4UjNyC3Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Extend freader with a flag specifying whether it's OK to cause page fault to fetch file data that is not already physically present in memory. With this, it's now easy to wait for data if the caller is running in sleepable (faultable) context. We utilize read_cache_folio() to bring the desired file page into page cache, after which the rest of the logic works just the same at page level. Suggested-by: Omar Sandoval Cc: Shakeel Butt Cc: Johannes Weiner Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 49 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 5f898fee43d7..23bfc811981a 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -20,6 +20,7 @@ struct freader { struct page *page; void *page_addr; u64 file_off; + bool may_fault; }; struct { const char *data; @@ -29,12 +30,13 @@ struct freader { }; static void freader_init_from_file(struct freader *r, void *buf, u32 buf_sz, - struct address_space *mapping) + struct address_space *mapping, bool may_fault) { memset(r, 0, sizeof(*r)); r->buf = buf; r->buf_sz = buf_sz; r->mapping = mapping; + r->may_fault = may_fault; } static void freader_init_from_mem(struct freader *r, const char *data, u64 data_sz) @@ -60,6 +62,17 @@ static int freader_get_page(struct freader *r, u64 file_off) freader_put_page(r); r->page = find_get_page(r->mapping, pg_off); + + if (!r->page && r->may_fault) { + struct folio *folio; + + folio = read_cache_folio(r->mapping, pg_off, NULL, NULL); + if (IS_ERR(folio)) + return PTR_ERR(folio); + + r->page = folio_file_page(folio, pg_off); + } + if (!r->page) return -EFAULT; /* page not mapped */ @@ -270,18 +283,8 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si /* enough for Elf64_Ehdr, Elf64_Phdr, and all the smaller requests */ #define MAX_FREADER_BUF_SZ 64 -/* - * Parse build ID of ELF file mapped to vma - * @vma: vma object - * @build_id: buffer to store build id, at least BUILD_ID_SIZE long - * @size: returns actual build id size in case of success - * - * Assumes no page fault can be taken, so if relevant portions of ELF file are - * not already paged in, fetching of build ID fails. - * - * Return: 0 on success; negative error, otherwise - */ -int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +static int __build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, + __u32 *size, bool may_fault) { const Elf32_Ehdr *ehdr; struct freader r; @@ -292,7 +295,7 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, if (!vma->vm_file) return -EINVAL; - freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file->f_mapping); + freader_init_from_file(&r, buf, sizeof(buf), vma->vm_file->f_mapping, may_fault); /* fetch first 18 bytes of ELF header for checks */ ehdr = freader_fetch(&r, 0, offsetofend(Elf32_Ehdr, e_type)); @@ -320,6 +323,22 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, return ret; } +/* + * Parse build ID of ELF file mapped to vma + * @vma: vma object + * @build_id: buffer to store build id, at least BUILD_ID_SIZE long + * @size: returns actual build id size in case of success + * + * Assumes no page fault can be taken, so if relevant portions of ELF file are + * not already paged in, fetching of build ID fails. + * + * Return: 0 on success; negative error, otherwise + */ +int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) +{ + return __build_id_parse(vma, build_id, size, false /* !may_fault */); +} + /* * Parse build ID of ELF file mapped to VMA * @vma: vma object @@ -333,7 +352,7 @@ int build_id_parse_nofault(struct vm_area_struct *vma, unsigned char *build_id, */ int build_id_parse(struct vm_area_struct *vma, unsigned char *build_id, __u32 *size) { - return -EOPNOTSUPP; + return __build_id_parse(vma, build_id, size, true /* may_fault */); } /** From patchwork Tue Jul 9 20:42:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCB9BC3DA45 for ; Tue, 9 Jul 2024 20:43:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65BC76B00A0; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60BE66B00A1; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D2EE6B00A2; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2D91E6B00A0 for ; Tue, 9 Jul 2024 16:43:15 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E115641A3A for ; Tue, 9 Jul 2024 20:43:14 +0000 (UTC) X-FDA: 82321389108.20.7CA4D3E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 4DCE340029 for ; Tue, 9 Jul 2024 20:43:13 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tDy3fCoO; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557758; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/T1RhSNFogjXWjH1T1rWHz4xvKilsEtvOIoqPPlohSc=; b=fYtKuHpGHE0aDt/biEFtdntm2UP551dcf8Y1+YqccSaxHJRMZdHn6R6VO9Y5DauKcomrzR xy7bPzw19gjAGoW2gCe3BxJfC+dF2sowLa1Wu57vm5IxnBNNQ8vjmI7guOJ3At1Flshgay LrfR/6iU1iiBoi4fwX2a6oEhdUSkYfs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557758; a=rsa-sha256; cv=none; b=abWnLypCdQWny5WC8/Mmj8AAc4h+zqYa1khW0BW45xpLDC9YTDlXivPsoOht5uFRSnOk6e 1V5g1nOTPZardSEyCYUwDiS1mnerAKJtRgDxHnkq1kq8oA7spyVKtV3ElNnu6Y1v5Y7Z91 VtFWeDmkPpTfI9j/M/jEFNBEi2oFFPU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tDy3fCoO; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of andrii@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=andrii@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0287561491; Tue, 9 Jul 2024 20:43:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87E1EC3277B; Tue, 9 Jul 2024 20:43:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557791; bh=z2GiV9WjwgN2ZnoTOoBHdWopafhvx9MuPC6sEC6Crd8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tDy3fCoO6SHfaItCzW9k8CB8fBKXCs1BTlMP2WzJw0Fs+wUdyGhbkQHRpAw2ChWsx PRrd5MiymWcZG0I/t0RJcTbpdUHKqRfUQNjfIY4+zt0VVLsjFiTFCoPbHNyJ32cYSu 2mtfAkldPT3hZ03K66B/JV/pK8E+zeW4kfqqUogOk94XGfrE1siXmMecEByhGf4Qy/ Elrl4rMytkXAQE56K2FJjXo+rv68tM8ZnwKH1ZmesV3BwnO6vy8zo351XHmryBi/0b bG2A0ILRlmFyPcKGUk+QWYQ3fg7ISLwYytUMqbfoZCVCPeD0WKHlq3pbT0fu59vPCK DKtb+WC5pg2RQ== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 06/10] lib/buildid: don't limit .note.gnu.build-id to the first page in ELF Date: Tue, 9 Jul 2024 13:42:41 -0700 Message-ID: <20240709204245.3847811-7-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4DCE340029 X-Stat-Signature: skyzqzqh785j5k869rdymgr17c6541dx X-Rspam-User: X-HE-Tag: 1720557793-122930 X-HE-Meta: U2FsdGVkX1+jsG9i59fcPwV/BnDwvGn0qIQp0EoYo0fLmRi4LH/bLjNmtV34CyjR4fedy1kBBfOiJduAGalcKIQK7Ugtuh7GwLhcqXbVcBXuqGiZ5rielMbV14gX1301j6QeXUXAtk2vrrbaMMh4pOjKPMydacQoHBrGozL6QOAutxsRLqCX/DQ6wUGvf0DH9UJ4et+eCycpE52T3DQqrfhyN5FBf38iAa5ZgEkwSBSYhT6Av/6FkAHYcpGMRgX6MIQ9bPRoFZs5smhjdS94+K1fiuQBNosSlqO7fx+RARZI+/jQj01YoxQdjKfb5ERQlKvXeH9MoZTQq1ZT5wtn6eytSBOwNRvAb5IL2/9XwRoL41TwURL28zRCJ4HQj00g2vesQYWggez5DwTTyqWig10y8Q1wXb5jDk4durzeDakHTumvnfBWRWTdKD/7sRKH7tJFWWlJTmHbtl8NfbqVTJ5Wnf9KTiZYRftfZAWLDczNQW6d2ofU3jIHcxUgrudVyo6WTFZqtXlrK2CTVyxHKY/+tA3aViJvHmc7gg0B/3WeyHYJIGvktkjcyDD2gA5BdwkD8LOCU11BKrf2yuE0bakwC/pmjPNoberjcvawImc5mElNkW+5/ClYRq3wZEZaEh0L/Pemhn8FBO6haZQ3t1zwEfHLFYxedxZxxEaZeqTCloP5X/rqkN5KA9tjr0gBERUG9jZ7Y4TblxIlFasMQL2JM6fAO5Q9Jsae6Ehz2WqlNM/mn4pUpRaldAcufxm8hBSQN3/jSC07mPjaa7wggXIoIPpCzD+/8I6z01OFpQa1iUza5scU4ICgGKRErtJuopeDWdygH0nvNJUmkytUI3qFQ+sUQ3YtNQOEYQNcIhqVR4bGAx4HzqKWEMAnxFzP0moFCjBm4Cx2Tb7PzkE9XgFv0d3hNqf7EK/Z+GvWPCem98cQVxAjPhFzu7avhnYwvoMHH7z8MKNpKL0hdwX 0whuPHRM QcHxWmZOuJHsyyzifPuDuvmt9VmZ+wvlSzxkbhH+0qqu/PnlxxcrkGM9DKziKBNdBUeWb5M/zOnkmZkNUyi3Id/m26ooxWBOG6iobkKXlhBtPcp+0SVorK153ILZHZLHLsCTd2zsmSD9vjE5xVo2/EBuDqZwhZLFnqkShmlWkC5fqCqjux1XlYuoxNldBOlAVYmxAtYiHsE0eLyQur/8T0ByruGuO0iPjn4P31rUwYqUVUgo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With freader we don't need to restrict ourselves to a single page, so let's allow ELF notes to be at any valid position with the file. We also merge parse_build_id() and parse_build_id_buf() as now the only difference between them is note offset overflow, which makes sense to check in all situations. Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 28 +++++++--------------------- 1 file changed, 7 insertions(+), 21 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 23bfc811981a..419966d88cd5 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -152,9 +152,8 @@ static void freader_cleanup(struct freader *r) * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are * identical. */ -static int parse_build_id_buf(struct freader *r, - unsigned char *build_id, __u32 *size, - u64 note_offs, Elf32_Word note_size) +static int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *size, + u64 note_offs, Elf32_Word note_size) { const char note_name[] = "GNU"; const size_t note_name_sz = sizeof(note_name); @@ -163,6 +162,10 @@ static int parse_build_id_buf(struct freader *r, const Elf32_Nhdr *nhdr; const char *data; + /* check for overflow */ + if (note_offs + note_size < note_offs) + return -EINVAL; + while (note_offs + sizeof(Elf32_Nhdr) < note_end) { nhdr = freader_fetch(r, note_offs, sizeof(Elf32_Nhdr) + note_name_sz); if (!nhdr) @@ -199,23 +202,6 @@ static int parse_build_id_buf(struct freader *r, return -EINVAL; } -static inline int parse_build_id(struct freader *r, - unsigned char *build_id, - __u32 *size, - u64 note_start_off, - Elf32_Word note_size) -{ - /* check for overflow */ - if (note_start_off + note_size < note_start_off) - return -EINVAL; - - /* only supports note that fits in the first page */ - if (note_start_off + note_size > PAGE_SIZE) - return -EINVAL; - - return parse_build_id_buf(r, build_id, size, note_start_off, note_size); -} - /* Parse build ID from 32-bit ELF */ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *size) { @@ -369,7 +355,7 @@ int build_id_parse_buf(const void *buf, unsigned char *build_id, u32 buf_size) freader_init_from_mem(&r, buf, buf_size); - return parse_build_id_buf(&r, build_id, NULL, 0, buf_size); + return parse_build_id(&r, build_id, NULL, 0, buf_size); } #if IS_ENABLED(CONFIG_STACKTRACE_BUILD_ID) || IS_ENABLED(CONFIG_VMCORE_INFO) From patchwork Tue Jul 9 20:42:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEAF4C3DA42 for ; Tue, 9 Jul 2024 20:43:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 720526B00A4; Tue, 9 Jul 2024 16:43:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D04E6B00A5; Tue, 9 Jul 2024 16:43:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 570306B00A6; Tue, 9 Jul 2024 16:43:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3677F6B00A4 for ; Tue, 9 Jul 2024 16:43:21 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 01194A52D0 for ; Tue, 9 Jul 2024 20:43:20 +0000 (UTC) X-FDA: 82321389402.29.6AABC75 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf02.hostedemail.com (Postfix) with ESMTP id B2A6480011 for ; Tue, 9 Jul 2024 20:43:18 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FT3VeWnK; spf=pass (imf02.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/p3ZJQXLxqxZi0QU2VNC5Qt4CTeV7snF007YBX98Nf0=; b=ugPrBIskLbVhnkR2e+E5HsMJNKzV5NAPe1cyVPNnOn4ZqXdDtlzE/RYGttt16GF9ARIyf8 ksokArLfvHBW8bFL5uL9pXTcLU9j5rwgKdBFN1yPZeClEex4z1hRTkfpMwth3mPwtHc7ea Wb+NMIfaiZRTLZN4+eXFWDxGICo+NNk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557768; a=rsa-sha256; cv=none; b=J2yjHKzlr6oU9FAgU4YoKzDg6QHrQNA1BxJcIi1pTi4Sljr0KVYKs5DsEdpxest0951dGh O/PnCtNl+j5vdc0XGFgVNGew5Rz8vp27cjGraThMETDkwjXPwfCUSy81lE45fFOqOY6g5g /hfopweVjH+JUzo5jjGmaqD5j9i9/M0= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=FT3VeWnK; spf=pass (imf02.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id BA55ECE12E8; Tue, 9 Jul 2024 20:43:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CEC70C3277B; Tue, 9 Jul 2024 20:43:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557795; bh=ntgIIpn+tzkBcO08o2+RKeRyEZVireJUIPte90Dh4v8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FT3VeWnKtWaL7Gbbv2pnSaxMvBXc21UChw8rj1kgj/anWnhmDG4nwXOHcG+aOzxgb VotIDtFAfz/XE8izfTt3S08+En+tj7Zfcw6vU8EmRqR2ssaVUwDiyBucSYBmlVUNTu 0i3d27DXzO+uKIVH4TKmPAdzahRpg90xGw+9Yb/MYeHAahjEF9B38Rhu2/OHQsyp78 MyOiFFwfoKzGK+pGBwc1paG/8eI7Esy+Vv7SsCwHmMDh5UknwSbdUdKSF9JW7mFKBH STzwdtinTGuqEnzure0ARFMPb3GcBUlc/hW6nUtR+sNSg3hufnwOL2JBFQE3OnKUjr yGjixGGcAaPfg== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 07/10] lib/buildid: harden build ID parsing logic some more Date: Tue, 9 Jul 2024 13:42:42 -0700 Message-ID: <20240709204245.3847811-8-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: nqhbcecindkwwcqey3gpoqhohrb8syxm X-Rspamd-Queue-Id: B2A6480011 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720557798-317974 X-HE-Meta: U2FsdGVkX18r+TTLuIlN4Irq6Qyjy9HgCUo9XOisZp4eQRznfQoDuQQRVW5QzHPi85xB6AWBLEqND9FvWmPgqyRp5qFYnl2HdEVD2Mj9G+68lLhVS2k5v9KNTh2rvZqLb/HD7nFiGLy/Nwsu3N9QrXduyjcqAHSyoRbzU1ulxbLKpevWjH2XOsAjhKuD0jt7hynzJWt7SS4NDCEZUBBpXm8Fu6jsDrG2FoOFVgcs+199jAWVFXveN+iGmf4XL0MOZwjAyqCCnWRPxc6uyp9Rky4DjhYhB+oqvOkzT3+8Z4V6mzEqRTYJCMf5Q5hAXhNFreUfw9BApPc148dN49EgwZtlWp4mPgsfWHCFKMkWYV3W3vBtCKm839k7JwhcMr9wQpO0g8jABUlPnJp+o8rL1C2aX8RfWFXlzCxm+X7WnhgpZKGm4xM2loxn+JOKmVMXtBgchWOH0rDbNB/oPsQWwTqsbSQrGS0088s0i8pwpOMzD1lyFCOvky8Ci8YK13fT1EUImpr6WCziJb9+yAymvwFBfcQyezyxrgvLGH109ciOFtJWrUJceyJa0smoZM2z7LjvNEMhMBhtIA/NiF64oXxYhI+9xGlDyB2n4l5Tk7kghlTTWegD2oYFPJGkfrlwO26jOGrLmjlwkbS5hAuBFPkV7VxvG/qSEknCmP99paCwfeygr5XJMhnJ7tejW476PPFk0I+NcYJ6PbFFKaOIKSHtC0NmvAdoyb9Rq9/iAfcL9cgmj0vXlWQ2BlUnUkzDaw/qZ7LXjWfQ8t4chimN7jf7n978DIn8/eEBkFoEChKWDH5TFgwrLg8ZpaqNt9S+VcSfmBLIqexFUYxUmKqaLQt2BTB6CG+GuI4swmua2+2MwEZz6407jMqXjRd4u/AdYFeJUawmtqq0eWr8tXe98XtT6OIwDR20Pc6jimcGN1V0ozN5ywskVRuPVlMPKs6jBtLAYWyamkAQwo08LoA 5cTH0NCu jsmCG1jD9XOAbnlhGBlWLGIMxvulf/IKglSANYaFETR+EGvY0rRmC/QthiAO4E5xrvvMnniRqcBxoL68cCxzbWwBE1fLrb8iNqi6PnUvsHoSzYh5PJ0gkIHmA6aaIQTQUiQl29T1YUMQxnuf4q3gxHq4WVj7r280nIPyesZQ5OQ6q6tqOCwxLr34u8KZ0xCzNDxM96Pdbav8fRszmxZrcKK5+0L2gnNG7XINpmg7V1FJSjphsHthkDpBU0p8dRAVJx0Gq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Harden build ID parsing logic some more, adding explicit READ_ONCE() when fetching values that we then use to check correctness and various note iteration invariants. Suggested-by: Andi Kleen Signed-off-by: Andrii Nakryiko --- lib/buildid.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/lib/buildid.c b/lib/buildid.c index 419966d88cd5..7e36a32fbb90 100644 --- a/lib/buildid.c +++ b/lib/buildid.c @@ -158,7 +158,7 @@ static int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *siz const char note_name[] = "GNU"; const size_t note_name_sz = sizeof(note_name); u64 build_id_off, new_offs, note_end = note_offs + note_size; - u32 build_id_sz; + u32 build_id_sz, name_sz, desc_sz; const Elf32_Nhdr *nhdr; const char *data; @@ -171,14 +171,15 @@ static int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *siz if (!nhdr) return r->err; - if (nhdr->n_type == BUILD_ID && - nhdr->n_namesz == note_name_sz && - !strcmp((char *)(nhdr + 1), note_name) && - nhdr->n_descsz > 0 && - nhdr->n_descsz <= BUILD_ID_SIZE_MAX) { + name_sz = READ_ONCE(nhdr->n_namesz); + desc_sz = READ_ONCE(nhdr->n_descsz); + if (READ_ONCE(nhdr->n_type) == BUILD_ID && + name_sz == note_name_sz && + !strncmp((char *)(nhdr + 1), note_name, note_name_sz) && + desc_sz > 0 && desc_sz <= BUILD_ID_SIZE_MAX) { build_id_off = note_offs + sizeof(Elf32_Nhdr) + ALIGN(note_name_sz, 4); - build_id_sz = nhdr->n_descsz; + build_id_sz = desc_sz; /* freader_fetch() will invalidate nhdr pointer */ data = freader_fetch(r, build_id_off, build_id_sz); @@ -192,8 +193,7 @@ static int parse_build_id(struct freader *r, unsigned char *build_id, __u32 *siz return 0; } - new_offs = note_offs + sizeof(Elf32_Nhdr) + - ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4); + new_offs = note_offs + sizeof(Elf32_Nhdr) + ALIGN(name_sz, 4) + ALIGN(desc_sz, 4); if (new_offs <= note_offs) /* overflow */ break; note_offs = new_offs; @@ -214,7 +214,7 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si return r->err; /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ - phnum = ehdr->e_phnum; + phnum = READ_ONCE(ehdr->e_phnum); phoff = READ_ONCE(ehdr->e_phoff); /* set upper bound on amount of segments (phdrs) we iterate */ @@ -226,8 +226,9 @@ static int get_build_id_32(struct freader *r, unsigned char *build_id, __u32 *si if (!phdr) return r->err; - if (phdr->p_type == PT_NOTE && - !parse_build_id(r, build_id, size, phdr->p_offset, phdr->p_filesz)) + if (READ_ONCE(phdr->p_type) == PT_NOTE && + !parse_build_id(r, build_id, size, + READ_ONCE(phdr->p_offset), READ_ONCE(phdr->p_filesz))) return 0; } return -EINVAL; @@ -246,7 +247,7 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si return r->err; /* subsequent freader_fetch() calls invalidate pointers, so remember locally */ - phnum = ehdr->e_phnum; + phnum = READ_ONCE(ehdr->e_phnum); phoff = READ_ONCE(ehdr->e_phoff); /* set upper bound on amount of segments (phdrs) we iterate */ @@ -258,8 +259,9 @@ static int get_build_id_64(struct freader *r, unsigned char *build_id, __u32 *si if (!phdr) return r->err; - if (phdr->p_type == PT_NOTE && - !parse_build_id(r, build_id, size, phdr->p_offset, phdr->p_filesz)) + if (READ_ONCE(phdr->p_type) == PT_NOTE && + !parse_build_id(r, build_id, size, + READ_ONCE(phdr->p_offset), READ_ONCE(phdr->p_filesz))) return 0; } From patchwork Tue Jul 9 20:42:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1165BC2BD09 for ; Tue, 9 Jul 2024 20:43:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 942496B00A5; Tue, 9 Jul 2024 16:43:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CAFB6B00A6; Tue, 9 Jul 2024 16:43:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76C816B00A7; Tue, 9 Jul 2024 16:43:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5691F6B00A5 for ; Tue, 9 Jul 2024 16:43:24 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 102261A1A49 for ; Tue, 9 Jul 2024 20:43:24 +0000 (UTC) X-FDA: 82321389528.29.7088C4A Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf01.hostedemail.com (Postfix) with ESMTP id BB5AA4001D for ; Tue, 9 Jul 2024 20:43:21 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OWv8PMY6; spf=pass (imf01.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557787; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gsmzGpKK8QXJw069hvnjOrZZkhs0zSJy4R5UDJ6ztho=; b=Ke7//cbEghLr4YvVRmxlN1A0n/Ur5NUyE2LL1dTMGlXM1tJv+W/wDEFhJVbI1Bsh0CwAN4 hMhZga5iFgC5R2Z1NFrGbkiDWmie7NVtGISPJDglL7W1EhXLs1UTnpeZ607zTPSxZI/Bz3 ZWWNPPAkZUzoBo6aSNBqrzZigXQfGRQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=OWv8PMY6; spf=pass (imf01.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557787; a=rsa-sha256; cv=none; b=ebFn2t6eDfC09haYjiQd4KPFa60C7IxHdsejS/umZ5xGAf1DaAr9gBprtNLt2m52n6cyPq fXRqfQtJ+F/5r1y3UJAlZQbagLSJAzwnWb603X7LQuKTjWjhOk/qpg9UL+2+SrJS3zFgc+ Sk029SDzNvxa90C1I7S0kCxkx7ZqNMg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 14B4DCE130A; Tue, 9 Jul 2024 20:43:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21B81C4AF0B; Tue, 9 Jul 2024 20:43:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557798; bh=kcQeVuVwnIJJkBHc/ynMBCpPQQtY5kY6QBTEhCYYFCI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OWv8PMY6JEthzTzEwmP72e2DJAJOPaEwZAp7O4/fr0gT4l5X7qaTvQu3XTg4YIpdQ lMR7sM7uLJABy0s5ECMyu6ejI7msl8sdp+Eeu1fSIOFIaIWCoCNisri6EHyDysYv9O o920CW+s/KmvV054Pih5VtsyfsH90njm7nTcAep0DTi5xVKu2+oIj/fMKU/JdXjsKx VS1gKIxrPZuZ+aC0X2h8WoWopAhqzcNH02v1154WIwnjH9g2yK2NWxSiO8Vg1TAb90 8xip0AgsUPfI2frI7jwdw9sq+nllMMmi1OMXrgJiELAit4vboPOj6tnDn7RlekKP6T pqcgIxFZsiVzw== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 08/10] bpf: decouple stack_map_get_build_id_offset() from perf_callchain_entry Date: Tue, 9 Jul 2024 13:42:43 -0700 Message-ID: <20240709204245.3847811-9-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: BB5AA4001D X-Stat-Signature: y1aa64t5esm7kuc7qhagwdd5q4mp648o X-HE-Tag: 1720557801-468947 X-HE-Meta: U2FsdGVkX19tNYT3wupSLsfuRDegw7HSmEDCNh//3i7c972i0wxVvFYZy7ZV2fE9ogISfKtdPhTzrGug8p8VOL9jXdf6fD5f7t32uWqbPf7QwqsOmSC3cYkvTnORWwwuTNXd1Vs385cU3TmtmpNMPvKyVuk063vtL4GD8QbxKVPytRC9HL4VLjm4nfa0NtQtp+CZTsbH7rnYZbmnqYyP0PYK5nL1ocAEX+3rns0cSfU0ocERZNl+zQyyZMo2O/LJm7IF5kdGlIqMTTCHSK6bqUehTyw+8+ufAYQewW4T2nADZHTD67fIMkcRfcM/7IMtfK1tQXMNnHLs0eI7MEG3Z8P+wRujx3RgCKjOOnIzXKxhXoBhK2RptiUi9Qm9isYz0C7uHRqtQcviP747cw/4LuNf4+uioZFYr00Mnp27U5XPkj8LQjyRPakNzkrxwQ53ipkOr1lbn57Z2BV9ayEtARBN29m+Hl0tJ55qmw5aTADEyD2ItN3s3AMlqqlhUJLThMLZamo3yqKoPoFfzlKVGvU3wA/4/j5P4NJA9sm4ZHoK7dBGFRyj2ei10LSVc6obXBi/CMRY5jjIXBGWbAxQby+uFkwV2Avb4KZX7FpD9SxeGJMddDDf2D9oLp+HhMKCqmcNE7oG3BTQAzr93YQ5l8AQcQQPymOYUyq4LUan695Hd6VrJO465i9IgZSSJq2CO6DON1uvYAIsOmz5yttwRgvbgFlsTkmLnvQ53rm2pnKUmSE4ZamUhYNM5DYWMalrzolNWfV1gUV5k9AvA2Jq2WBXfXAkM8aIA2WeylhUxb/unk357U4NdX6PXzeGJMZqRi6Vfz5qVBCNoX6FHEbhM/v1NNF6MXK/n2ODXAMimCt2qczWKJECBn4zN6yRKuypFaTlNuBHxhiuDF8ePQcl8ALDO9KJH9ZghEJl96hgJr4o/+Vdv+KC7DmD2HKOkGMb2tGfMoGozltXyxYQM8O oc5K8+LM ohDnN9J+1E2bCLmit9CYjMl0iQVkOr23WP8ZiAVpEdyLFfLPJP+GQQ5YjMGa1lBAQQE6zCDpaoQopL0NOR40Ah809kD+yacpqZ/sHF4ECpI9dDCk13mmzs1/tYsBVN6f4VnoWPp6CcnD0QB/FnjHFSLH65Xfh0e6wL/kivkRp8a8xHEV1Z2gI+9hy8Ef7GSHpPEkyh6Vh8Wn+ej5MvUiFbilbBq0+zlW5P8TiWg4H3VVFr1A4LySa82vM30MEd4tuq+iQWvpc9VMcmLEPAog++6vBZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change stack_map_get_build_id_offset() which is used to convert stack trace IP addresses into build ID+offset pairs. Right now this function accepts an array of u64s as an input, and uses array of struct bpf_stack_build_id as an output. This is problematic because u64 array is coming from perf_callchain_entry, which is (non-sleepable) RCU protected, so once we allows sleepable build ID fetching, this all breaks down. But its actually pretty easy to make stack_map_get_build_id_offset() works with array of struct bpf_stack_build_id as both input and output. Which is what this patch is doing, eliminating the dependency on perf_callchain_entry. We require caller to fill out bpf_stack_build_id.ip fields (all other can be left uninitialized), and update in place as we do build ID resolution. We make sure to READ_ONCE() and cache locally current IP value as we used it in a few places to find matching VMA and so on. Given this data is directly accessible and modifiable by user's BPF code, we should make sure to have a consistent view of it. Signed-off-by: Andrii Nakryiko --- kernel/bpf/stackmap.c | 49 +++++++++++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 770ae8e88016..6457222b0b46 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -124,8 +124,18 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(err); } +/* + * Expects all id_offs[i].ip values to be set to correct initial IPs. + * They will be subsequently: + * - either adjusted in place to a file offset, if build ID fetching + * succeeds; in this case id_offs[i].build_id is set to correct build ID, + * and id_offs[i].status is set to BPF_STACK_BUILD_ID_VALID; + * - or IP will be kept intact, if build ID fetching failed; in this case + * id_offs[i].build_id is zeroed out and id_offs[i].status is set to + * BPF_STACK_BUILD_ID_IP. + */ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, - u64 *ips, u32 trace_nr, bool user) + u32 trace_nr, bool user) { int i; struct mmap_unlock_irq_work *work = NULL; @@ -142,30 +152,28 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, /* cannot access current->mm, fall back to ips */ for (i = 0; i < trace_nr; i++) { id_offs[i].status = BPF_STACK_BUILD_ID_IP; - id_offs[i].ip = ips[i]; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); } return; } for (i = 0; i < trace_nr; i++) { - if (range_in_vma(prev_vma, ips[i], ips[i])) { + u64 ip = READ_ONCE(id_offs[i].ip); + + if (range_in_vma(prev_vma, ip, ip)) { vma = prev_vma; - memcpy(id_offs[i].build_id, prev_build_id, - BUILD_ID_SIZE_MAX); + memcpy(id_offs[i].build_id, prev_build_id, BUILD_ID_SIZE_MAX); goto build_id_valid; } - vma = find_vma(current->mm, ips[i]); + vma = find_vma(current->mm, ip); if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; - id_offs[i].ip = ips[i]; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); continue; } build_id_valid: - id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ips[i] - - vma->vm_start; + id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ip - vma->vm_start; id_offs[i].status = BPF_STACK_BUILD_ID_VALID; prev_vma = vma; prev_build_id = id_offs[i].build_id; @@ -216,7 +224,7 @@ static long __bpf_get_stackid(struct bpf_map *map, struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map); struct stack_map_bucket *bucket, *new_bucket, *old_bucket; u32 skip = flags & BPF_F_SKIP_FIELD_MASK; - u32 hash, id, trace_nr, trace_len; + u32 hash, id, trace_nr, trace_len, i; bool user = flags & BPF_F_USER_STACK; u64 *ips; bool hash_matches; @@ -238,15 +246,18 @@ static long __bpf_get_stackid(struct bpf_map *map, return id; if (stack_map_use_build_id(map)) { + struct bpf_stack_build_id *id_offs; + /* for build_id+offset, pop a bucket before slow cmp */ new_bucket = (struct stack_map_bucket *) pcpu_freelist_pop(&smap->freelist); if (unlikely(!new_bucket)) return -ENOMEM; new_bucket->nr = trace_nr; - stack_map_get_build_id_offset( - (struct bpf_stack_build_id *)new_bucket->data, - ips, trace_nr, user); + id_offs = (struct bpf_stack_build_id *)new_bucket->data; + for (i = 0; i < trace_nr; i++) + id_offs[i].ip = ips[i]; + stack_map_get_build_id_offset(id_offs, trace_nr, user); trace_len = trace_nr * sizeof(struct bpf_stack_build_id); if (hash_matches && bucket->nr == trace_nr && memcmp(bucket->data, new_bucket->data, trace_len) == 0) { @@ -445,10 +456,16 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, copy_len = trace_nr * elem_size; ips = trace->ip + skip; - if (user && user_build_id) - stack_map_get_build_id_offset(buf, ips, trace_nr, user); - else + if (user && user_build_id) { + struct bpf_stack_build_id *id_offs = buf; + u32 i; + + for (i = 0; i < trace_nr; i++) + id_offs[i].ip = ips[i]; + stack_map_get_build_id_offset(buf, trace_nr, user); + } else { memcpy(buf, ips, copy_len); + } if (size > copy_len) memset(buf + copy_len, 0, size - copy_len); From patchwork Tue Jul 9 20:42:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64138C2BD09 for ; Tue, 9 Jul 2024 20:43:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC4206B00A7; Tue, 9 Jul 2024 16:43:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E21FF6B00A8; Tue, 9 Jul 2024 16:43:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA0646B00A9; Tue, 9 Jul 2024 16:43:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A8C646B00A7 for ; Tue, 9 Jul 2024 16:43:27 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7078C41A71 for ; Tue, 9 Jul 2024 20:43:27 +0000 (UTC) X-FDA: 82321389654.04.FE15FC7 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf21.hostedemail.com (Postfix) with ESMTP id 2AE1F1C0005 for ; Tue, 9 Jul 2024 20:43:24 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Wt028Hqg; spf=pass (imf21.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557781; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=14G0hq5G50msPn1uoqbR1+8SYDlRBodKmLLoY7TY2RA=; b=PJEELgw6gOqoC7tE/5rtXYMwGEW5mELybIeLJtUnCpY9NKGmp0I9aU1OR33OVXXrG0UxNt r+FuVMAmISDngLtFkWD3LTiK9Ysm8IQfe7I/mXhWcr3VRYs9Wy9vbJ/oUcF2IcV+2f89eU VnAqcktybTOOqNBhOWKPQzY/sxfQhOw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Wt028Hqg; spf=pass (imf21.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557781; a=rsa-sha256; cv=none; b=0+CozSz1Jc6CAtLs1bDa/c03jcNocnq7YCg81ONEtDDZq994EavLjMAqKbraE0xxNYHY1A E/guUKNBMPvdVh5nHdCE11VrUPnneWrqdEI3GTe1aFjkzqW1uE0fJ8k70XWtaOw8Jgch2U +sgPXxdWDEs1hYUEqaLPfx/gbNibXRo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 5F96DCE12B9; Tue, 9 Jul 2024 20:43:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 654B3C3277B; Tue, 9 Jul 2024 20:43:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557801; bh=1A5WSb+MzjCnjVF4/c2tdeCraVnxcv1jVhPhwXpE8lU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wt028HqgxLOKt5zz/lK5ao7VAcxkUmwgMpwgEwMPvEEggwl59q8jO5svpZJ8hbkFx BKaGg0vmaFz5Fly9J0B7bod4W4EizaXdLrK344fPeT2T36LrHegYqWAI/zF1OJ9ww9 ctWeICRmp9npg26uia5KwNvAiZI/m3zVaVtot/I1GY1Hkj5iCWrhPuJGbBEa8zqIPK wUKPflZAtvwzs99iJ+zTrQUlA6khFJr4Sf22H44C1J+f4oSzkbESLP+2EqNMK1yYvo jZye98EkiT4jw74KvDO01IQhrCVunu+95I3qVKbvqHsEwT0wRUZXDKpsOtHATRLEK4 AZ8CDvPhjFMtw== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 09/10] bpf: wire up sleepable bpf_get_stack() and bpf_get_task_stack() helpers Date: Tue, 9 Jul 2024 13:42:44 -0700 Message-ID: <20240709204245.3847811-10-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Stat-Signature: dhqk5ca7r55aq4mm5dd1tpm1pu34beuf X-Rspam-User: X-Rspamd-Queue-Id: 2AE1F1C0005 X-Rspamd-Server: rspam02 X-HE-Tag: 1720557804-83021 X-HE-Meta: U2FsdGVkX19o9BXDWMgcQZKOBtXHTQt4l5tquqY1p6tEPncas5z/kGimLwMbHJl1X9D9L0I1kMtfKoj20IqRyrCahN3ijGtsGbpJPGfBLZGLdlvrd7gHYzyO7DPmSxArbq/3IuZmiX67Lm7LkXhCZDEsJ36oIxFyFiVfxaVxXiTxyAhP1eioTyYsokJOkDZjtgyDxrIN9AmreWGaNuqaNZtI1Ca4f0PqWB/wgYacDTa6i/NRp5/ySc+RLf9pTCjghNgEQlytVED3wIpRBh1DXxA2cI2YZTjziyGU97SpTK/FXMk6O8SLnSYbJOlUQHsW64Cnpzzk2r/ONuZKI3goaQWRifBPsYZJ8mawKqH3bVjaYf9NVbxs9X+xvkMlbQ1fdTnRRXp3s7x5tXztetBUuFGSTcqzmNX/7eDfU93B7bJit5kU2jJEz5M3wHsX11nJjwdWMf/GYrGM/C0IXaQLrliHQFCyIR7nQ6hrDhZ/bQuw1ueqJlfOTzFdRqnOzr0nygvy/dLzLphdVCUuDcUR7DxOJ7WWiGU/4Plzr2GJbW86bDEmj8VDcsvfFsBnOynLFO77LpHmgs6ERtzmLNvb53fdhObwZbhOb0r/vEfCtoIdrI90VKzpUvvSBtqIc3E/7CCMP2HHzynDwdlGElB9+xwEzibtJFWymD9HYMIwoP/1PTlK0XpW0OkYlTaqCBYDKSwGLscUVJI48VSTfZL4CSmq6mYJDFw6udYMohZ6fBLLRe3FRucqncuLv3oWiCulVzytkeOn85eIrIaxFpujTTNtk9bsFP0Sa1RwtYXaVRW0GxyRLi5CnIonG2TglWt6TEHZADzTh4MshcIQJS1HU7T4lifDX45CmFMT1Jt5PDw+5Xg9JijMUMMuDe98VJsiUZndiCVT082WBHRO+MYqKLtmAKZ8sbw5HSVe64Oqub5Z00+Hqh2JYpoZ61k5p+FeauVOv5ZHaJb2CtQtiF7 b/BxUCFa TTUN2G1RYiCQ80qVXo7Gq9HsrzN98/HJDeGitfOqS8USQeqWyb2F93+K+GEWri+Jt69spCnzlM/sGtmRRZ6PpDxQE1ywVKcyu3fHq7f7M+mrV5Q4voRGe9gZO8qAXl3wjlXmKpQ60lPxRyM5xKKh8USgwjW3un9Avcq3MiJFdVwR2vkh9p7GONZWzvW26cccoOUxOvjKwGoIX7uwHVU9fXuUSvJipGr/gkGLMAPXy/BCqBYhlrX/QmSnSdlUGCTVpaKCijjT1tziri4vMTSy0MTi9ww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add sleepable implementations of bpf_get_stack() and bpf_get_task_stack() helpers and allow them to be used from sleepable BPF program (e.g., sleepable uprobes). Note, the stack trace IPs capturing itself is not sleepable (that would need to be a separate project), only build ID fetching is sleepable and thus more reliable, as it will wait for data to be paged in, if necessary. For that we make use of sleepable build_id_parse() implementation. Now that build ID related internals in kernel/bpf/stackmap.c can be used both in sleepable and non-sleepable contexts, we need to add additional rcu_read_lock()/rcu_read_unlock() protection around fetching perf_callchain_entry, but with the refactoring in previous commit it's now pretty straightforward. We make sure to do rcu_read_unlock (in sleepable mode only) right before stack_map_get_build_id_offset() call which can sleep. By that time we don't have any more use of perf_callchain_entry. Note, bpf_get_task_stack() will fail for user mode if task != current. And for kernel mode build ID are irrelevant. So in that sense adding sleepable bpf_get_task_stack() implementation is a no-op. It feel right to wire this up for symmetry and completeness, but I'm open to just dropping it until we support `user && crosstask` condition. Signed-off-by: Andrii Nakryiko --- include/linux/bpf.h | 2 + kernel/bpf/stackmap.c | 90 ++++++++++++++++++++++++++++++++-------- kernel/trace/bpf_trace.c | 5 ++- 3 files changed, 77 insertions(+), 20 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 960780ef04e1..1c28bc9a34e2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -3182,7 +3182,9 @@ extern const struct bpf_func_proto bpf_get_current_uid_gid_proto; extern const struct bpf_func_proto bpf_get_current_comm_proto; extern const struct bpf_func_proto bpf_get_stackid_proto; extern const struct bpf_func_proto bpf_get_stack_proto; +extern const struct bpf_func_proto bpf_get_stack_sleepable_proto; extern const struct bpf_func_proto bpf_get_task_stack_proto; +extern const struct bpf_func_proto bpf_get_task_stack_sleepable_proto; extern const struct bpf_func_proto bpf_get_stackid_proto_pe; extern const struct bpf_func_proto bpf_get_stack_proto_pe; extern const struct bpf_func_proto bpf_sock_map_update_proto; diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 6457222b0b46..3615c06b7dfa 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -124,6 +124,12 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(err); } +static int fetch_build_id(struct vm_area_struct *vma, unsigned char *build_id, bool may_fault) +{ + return may_fault ? build_id_parse(vma, build_id, NULL) + : build_id_parse_nofault(vma, build_id, NULL); +} + /* * Expects all id_offs[i].ip values to be set to correct initial IPs. * They will be subsequently: @@ -135,7 +141,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) * BPF_STACK_BUILD_ID_IP. */ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, - u32 trace_nr, bool user) + u32 trace_nr, bool user, bool may_fault) { int i; struct mmap_unlock_irq_work *work = NULL; @@ -166,7 +172,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs, goto build_id_valid; } vma = find_vma(current->mm, ip); - if (!vma || build_id_parse_nofault(vma, id_offs[i].build_id, NULL)) { + if (!vma || fetch_build_id(vma, id_offs[i].build_id, may_fault)) { /* per entry fall back to ips */ id_offs[i].status = BPF_STACK_BUILD_ID_IP; memset(id_offs[i].build_id, 0, BUILD_ID_SIZE_MAX); @@ -257,7 +263,7 @@ static long __bpf_get_stackid(struct bpf_map *map, id_offs = (struct bpf_stack_build_id *)new_bucket->data; for (i = 0; i < trace_nr; i++) id_offs[i].ip = ips[i]; - stack_map_get_build_id_offset(id_offs, trace_nr, user); + stack_map_get_build_id_offset(id_offs, trace_nr, user, false /* !may_fault */); trace_len = trace_nr * sizeof(struct bpf_stack_build_id); if (hash_matches && bucket->nr == trace_nr && memcmp(bucket->data, new_bucket->data, trace_len) == 0) { @@ -398,7 +404,7 @@ const struct bpf_func_proto bpf_get_stackid_proto_pe = { static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, struct perf_callchain_entry *trace_in, - void *buf, u32 size, u64 flags) + void *buf, u32 size, u64 flags, bool may_fault) { u32 trace_nr, copy_len, elem_size, num_elem, max_depth; bool user_build_id = flags & BPF_F_USER_BUILD_ID; @@ -416,8 +422,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, if (kernel && user_build_id) goto clear; - elem_size = (user && user_build_id) ? sizeof(struct bpf_stack_build_id) - : sizeof(u64); + elem_size = user_build_id ? sizeof(struct bpf_stack_build_id) : sizeof(u64); if (unlikely(size % elem_size)) goto clear; @@ -438,6 +443,9 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, if (sysctl_perf_event_max_stack < max_depth) max_depth = sysctl_perf_event_max_stack; + if (may_fault) + rcu_read_lock(); /* need RCU for perf's callchain below */ + if (trace_in) trace = trace_in; else if (kernel && task) @@ -445,28 +453,35 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, else trace = get_perf_callchain(regs, 0, kernel, user, max_depth, crosstask, false); - if (unlikely(!trace)) - goto err_fault; - if (trace->nr < skip) + if (unlikely(!trace) || trace->nr < skip) { + if (may_fault) + rcu_read_unlock(); goto err_fault; + } trace_nr = trace->nr - skip; trace_nr = (trace_nr <= num_elem) ? trace_nr : num_elem; copy_len = trace_nr * elem_size; ips = trace->ip + skip; - if (user && user_build_id) { + if (user_build_id) { struct bpf_stack_build_id *id_offs = buf; u32 i; for (i = 0; i < trace_nr; i++) id_offs[i].ip = ips[i]; - stack_map_get_build_id_offset(buf, trace_nr, user); } else { memcpy(buf, ips, copy_len); } + /* trace/ips should not be dereferenced after this point */ + if (may_fault) + rcu_read_unlock(); + + if (user_build_id) + stack_map_get_build_id_offset(buf, trace_nr, user, may_fault); + if (size > copy_len) memset(buf + copy_len, 0, size - copy_len); return copy_len; @@ -481,7 +496,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struct task_struct *task, BPF_CALL_4(bpf_get_stack, struct pt_regs *, regs, void *, buf, u32, size, u64, flags) { - return __bpf_get_stack(regs, NULL, NULL, buf, size, flags); + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, false /* !may_fault */); } const struct bpf_func_proto bpf_get_stack_proto = { @@ -494,8 +509,24 @@ const struct bpf_func_proto bpf_get_stack_proto = { .arg4_type = ARG_ANYTHING, }; -BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, - u32, size, u64, flags) +BPF_CALL_4(bpf_get_stack_sleepable, struct pt_regs *, regs, void *, buf, u32, size, + u64, flags) +{ + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, true /* may_fault */); +} + +const struct bpf_func_proto bpf_get_stack_sleepable_proto = { + .func = bpf_get_stack_sleepable, + .gpl_only = true, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_CTX, + .arg2_type = ARG_PTR_TO_UNINIT_MEM, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +}; + +static long __bpf_get_task_stack(struct task_struct *task, void *buf, u32 size, + u64 flags, bool may_fault) { struct pt_regs *regs; long res = -EINVAL; @@ -505,12 +536,18 @@ BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, regs = task_pt_regs(task); if (regs) - res = __bpf_get_stack(regs, task, NULL, buf, size, flags); + res = __bpf_get_stack(regs, task, NULL, buf, size, flags, may_fault); put_task_stack(task); return res; } +BPF_CALL_4(bpf_get_task_stack, struct task_struct *, task, void *, buf, + u32, size, u64, flags) +{ + return __bpf_get_task_stack(task, buf, size, flags, false /* !may_fault */); +} + const struct bpf_func_proto bpf_get_task_stack_proto = { .func = bpf_get_task_stack, .gpl_only = false, @@ -522,6 +559,23 @@ const struct bpf_func_proto bpf_get_task_stack_proto = { .arg4_type = ARG_ANYTHING, }; +BPF_CALL_4(bpf_get_task_stack_sleepable, struct task_struct *, task, void *, buf, + u32, size, u64, flags) +{ + return __bpf_get_task_stack(task, buf, size, flags, true /* !may_fault */); +} + +const struct bpf_func_proto bpf_get_task_stack_sleepable_proto = { + .func = bpf_get_task_stack_sleepable, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_BTF_ID, + .arg1_btf_id = &btf_tracing_ids[BTF_TRACING_TYPE_TASK], + .arg2_type = ARG_PTR_TO_UNINIT_MEM, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +}; + BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, void *, buf, u32, size, u64, flags) { @@ -533,7 +587,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, __u64 nr_kernel; if (!(event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)) - return __bpf_get_stack(regs, NULL, NULL, buf, size, flags); + return __bpf_get_stack(regs, NULL, NULL, buf, size, flags, false /* !may_fault */); if (unlikely(flags & ~(BPF_F_SKIP_FIELD_MASK | BPF_F_USER_STACK | BPF_F_USER_BUILD_ID))) @@ -553,7 +607,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, __u64 nr = trace->nr; trace->nr = nr_kernel; - err = __bpf_get_stack(regs, NULL, trace, buf, size, flags); + err = __bpf_get_stack(regs, NULL, trace, buf, size, flags, false /* !may_fault */); /* restore nr */ trace->nr = nr; @@ -565,7 +619,7 @@ BPF_CALL_4(bpf_get_stack_pe, struct bpf_perf_event_data_kern *, ctx, goto clear; flags = (flags & ~BPF_F_SKIP_FIELD_MASK) | skip; - err = __bpf_get_stack(regs, NULL, trace, buf, size, flags); + err = __bpf_get_stack(regs, NULL, trace, buf, size, flags, false /* !may_fault */); } return err; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index cd098846e251..c3845470f56d 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1598,7 +1598,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_jiffies64: return &bpf_jiffies64_proto; case BPF_FUNC_get_task_stack: - return &bpf_get_task_stack_proto; + return prog->sleepable ? &bpf_get_task_stack_sleepable_proto + : &bpf_get_task_stack_proto; case BPF_FUNC_copy_from_user: return &bpf_copy_from_user_proto; case BPF_FUNC_copy_from_user_task: @@ -1654,7 +1655,7 @@ kprobe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) case BPF_FUNC_get_stackid: return &bpf_get_stackid_proto; case BPF_FUNC_get_stack: - return &bpf_get_stack_proto; + return prog->sleepable ? &bpf_get_stack_sleepable_proto : &bpf_get_stack_proto; #ifdef CONFIG_BPF_KPROBE_OVERRIDE case BPF_FUNC_override_return: return &bpf_override_return_proto; From patchwork Tue Jul 9 20:42:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 13728515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86AE4C2BD09 for ; Tue, 9 Jul 2024 20:43:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 264896B00A9; Tue, 9 Jul 2024 16:43:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2149A6B00AA; Tue, 9 Jul 2024 16:43:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DC366B00AB; Tue, 9 Jul 2024 16:43:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E41956B00A9 for ; Tue, 9 Jul 2024 16:43:30 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 885801C3817 for ; Tue, 9 Jul 2024 20:43:30 +0000 (UTC) X-FDA: 82321389780.06.3176A27 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 45F3B14000F for ; Tue, 9 Jul 2024 20:43:27 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=syDbLbkt; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720557779; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ago8UIKakUbx1w8xAWzA9HsZh/OpRn31XYZP6bTJG1M=; b=sctRf7QjOBUdXIXho8WjRcQHRJQBMAKiyuY8bovmb+ZBEnY52eV1ZD4g9EzWcubFokwDFO JgFmY6j1JZ3bGKb80ra5/GPqwaK85tmWb8nBdN+tj139ZSM+uuhZIPBSKf4izb01eA3xNh aWYm4nQIR9iJY3ALzEKWa8zBa86qgCc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720557779; a=rsa-sha256; cv=none; b=L9SaUxs5Mb9f3c5iJhss2B1F0ibbs1KDmN9GWO9xTruqwhw4GdTwebUGYDavsABcaET7NX fWJ211ilmQsnQ/+mFY6MIKvse91RN9A2J1NyaoPKVZGku8+xji3PUTYePcGfThDDafqZ9z 8rA/K0lrg5ffCndJxfFF3/FUCR3Dwk8= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=syDbLbkt; spf=pass (imf09.hostedemail.com: domain of andrii@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=andrii@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 97F74CE12E8; Tue, 9 Jul 2024 20:43:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4F7CC4AF0C; Tue, 9 Jul 2024 20:43:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720557804; bh=9f5XHZsGbbTDw89JnaMKZvno0TqLABUY0GcFo3N7UXE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=syDbLbkt/rrCN4xxHQDcqceSI4SU1mZuUzAg+FrJFKGOoHJXVVk6BzxxQyb5GfLdL ZdlePWJkPYUdWGBzoxMzOJLkTdDKgD1fKTAB0mWnHiDYsGwDHhMAqF5yTgX4sQWwqN Ml/mCbxu/ix7FsISj++JBIuFWxWemRvDWSc0MoplmJRQ00xdJzhicceBv7ZwpLJbjt io7XO6p0oxkXU+a6cfxADVumNjJtRseGb6Ijx/hwRLm0z0O8Xag8AJdCh19G8MjIDr Iqe47hAejf67sDNAe6tHVX4UC8ShaLWzm+gpDTYwvTAHFtiiiBCuW1yjSfWbQb//zy nJIuTmmXk9GkA== From: Andrii Nakryiko To: bpf@vger.kernel.org Cc: linux-mm@kvack.org, akpm@linux-foundation.org, adobriyan@gmail.com, shakeel.butt@linux.dev, hannes@cmpxchg.org, ak@linux.intel.com, osandov@osandov.com, Andrii Nakryiko Subject: [PATCH bpf-next 10/10] selftests/bpf: add build ID tests Date: Tue, 9 Jul 2024 13:42:45 -0700 Message-ID: <20240709204245.3847811-11-andrii@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240709204245.3847811-1-andrii@kernel.org> References: <20240709204245.3847811-1-andrii@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 45F3B14000F X-Stat-Signature: gei83nuzuqrzt3mzrodpjo99e9qqxka6 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720557807-603135 X-HE-Meta: U2FsdGVkX18KfLt1AqM1ZufXuDMLX1phyT+kd6Jz9VLQFCF8oTdKv3ehFj2L8kUOsR0D9P8Z/E6rVxPdIsr3yv+fQGjJqTheN8aKvJTyw5ZdLewWzgyDUdDJj7zyI2/q0DYOCxUSrLIv8YJ+oEYaWttkc9AnRNobCJkFJdoackuNUZ0OmugDIp2CfILTIifgzlbj0fc2RE1NFNrSiz/ZDv6yvhOcWUuGnlSmE8UlaF2qtEwzJERG0Cu4rcFlrObqDfMnrOzTzLhDMYt+Sf+GtSqnDA7wateE7PGzA+rJbXyk/pOIP2VRcZzdR4wtShuJ9OJh+H67xzkMn7KW1JOSEcd7YegwwNOON+zZjJeCa38SukNjnWt4hfH/x7jd5XrOcSkGSD3ad2o1vm5oy6FTHnR2+a7+ddIPO1/HUbDDkuKmmClaS3t0iRvou7s/DHUPSFsCO4nCRUvy7WAAaqR+a+ejw5hMe8TuqUCVVboIzUZYuPcOFiHmmHjGDmoU6ZxlFdLhvJ9yV7VF981+l+unPb4mfdH0U5KITaKbxyXSFVlhEarQjNr/TPxfL1XqRYmFHN+lGBmIyzPabgNpZ8qO3tWL2egZ3F/6gkP9Pd5JlvX+/OcO9/S26nOy4T03YCX2V7ip8BUapz5mm8/3kzT2z8ZTJ7edC9W7xSLWsMrzPlYDeyzbdD2YskVWuni40in9l6qe/d9UMk8ztnjkflOUbQHG9VtpF3LK7cM88ILGXJHBat//XNxTENQpTdtD2ziOv6X7Cpd+fW2oPvLaKMHDaejxaOjBPQA/qFfcYc1adogrDDwGmBStBxmHlGB51jzTqRMK3WsZr9Ln64cAyvONbD4J4C78lqKOXAtrYOTJnOgWnZgSyPCsKQIYKaVUxt7efOuHGa2Rt3HVM+DSNYTVwNf604fvt1nP3hV/vE2p3nbJ5u81GLYJm0CMEG58cKlXUkU1mmCVTIE5ewTRab2 H+Lp0QDi c6QdPiOTKdWL1cnQkBbIgmY/SLCOQFTQMTeS1jGHjRvbukd0HfgVBo3OdTqmJ9fXNTLK5EXqUI9C9iI3fOrZPbaxuANDn67l3QkPVN9aLORjm0suexy2GsUzWD5AaPlsWQvJbWBER75HIhghOQmk/pBlcS3J2nPf6/YYpiY3muTaRSQgr+ysO3pwHKxlYHtMX1TTQBmkgQomM7o8GVcYUA0K1vdqYIHKYnNDfdauxknTlo8s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a new set of tests validating behavior of capturing stack traces with build ID. We extend uprobe_multi target binary with ability to trigger uprobe (so that we can capture stack traces from it), but also we allow to force build ID data to be either resident or non-resident in memory. That way we can validate that in non-sleepable context we won't get build ID (as expected), but with sleepable uprobes we will get that build ID regardless of it being physically present in memory. Also, we add a small add-on linker script which reorders .note.gnu.build-id section and puts it after (big) .text section, putting build ID data outside of the very first page of ELF file. This will test all the relaxations we did in build ID parsing logic in kernel thanks to freader abstraction. Signed-off-by: Andrii Nakryiko --- tools/testing/selftests/bpf/Makefile | 5 +- .../selftests/bpf/prog_tests/build_id.c | 118 ++++++++++++++++++ .../selftests/bpf/progs/test_build_id.c | 31 +++++ tools/testing/selftests/bpf/uprobe_multi.c | 34 +++++ tools/testing/selftests/bpf/uprobe_multi.ld | 11 ++ 5 files changed, 197 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/build_id.c create mode 100644 tools/testing/selftests/bpf/progs/test_build_id.c create mode 100644 tools/testing/selftests/bpf/uprobe_multi.ld diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index e0b3887b3d2d..45f67e822f49 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -762,9 +762,10 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o $(call msg,BINARY,,$@) $(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@ -$(OUTPUT)/uprobe_multi: uprobe_multi.c +$(OUTPUT)/uprobe_multi: uprobe_multi.c uprobe_multi.ld $(call msg,BINARY,,$@) - $(Q)$(CC) $(CFLAGS) -O0 $(LDFLAGS) $^ $(LDLIBS) -o $@ + $(Q)$(CC) $(CFLAGS) -Wl,-T,uprobe_multi.ld -O0 $(LDFLAGS) \ + $(filter-out %.ld,$^) $(LDLIBS) -o $@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ diff --git a/tools/testing/selftests/bpf/prog_tests/build_id.c b/tools/testing/selftests/bpf/prog_tests/build_id.c new file mode 100644 index 000000000000..8e6d3603be61 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/build_id.c @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include + +#include "test_build_id.skel.h" + +static char build_id[BPF_BUILD_ID_SIZE]; +static int build_id_sz; + +static void print_stack(struct bpf_stack_build_id *stack, int frame_cnt) +{ + int i, j; + + for (i = 0; i < frame_cnt; i++) { + printf("FRAME #%02d: ", i); + switch (stack[i].status) { + case BPF_STACK_BUILD_ID_EMPTY: + printf("\n"); + break; + case BPF_STACK_BUILD_ID_VALID: + printf("BUILD ID = "); + for (j = 0; j < BPF_BUILD_ID_SIZE; j++) + printf("%02hhx", (unsigned)stack[i].build_id[j]); + printf(" OFFSET = %llx", (unsigned long long)stack[i].offset); + break; + case BPF_STACK_BUILD_ID_IP: + printf("IP = %llx", (unsigned long long)stack[i].ip); + break; + default: + printf("UNEXPECTED STATUS %d ", stack[i].status); + break; + } + printf("\n"); + } +} + +static void subtest_nofault(bool build_id_resident) +{ + struct test_build_id *skel; + struct bpf_stack_build_id *stack; + int frame_cnt; + + skel = test_build_id__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + skel->links.uprobe_nofault = bpf_program__attach(skel->progs.uprobe_nofault); + if (!ASSERT_OK_PTR(skel->links.uprobe_nofault, "link")) + goto cleanup; + + if (build_id_resident) + ASSERT_OK(system("./uprobe_multi uprobe-paged-in"), "trigger_uprobe"); + else + ASSERT_OK(system("./uprobe_multi uprobe-paged-out"), "trigger_uprobe"); + + if (!ASSERT_GT(skel->bss->res_nofault, 0, "res")) + goto cleanup; + + stack = skel->bss->stack_nofault; + frame_cnt = skel->bss->res_nofault / sizeof(struct bpf_stack_build_id); + if (env.verbosity >= VERBOSE_NORMAL) + print_stack(stack, frame_cnt); + + if (build_id_resident) { + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_VALID, "build_id_status"); + ASSERT_EQ(memcmp(stack[0].build_id, build_id, build_id_sz), 0, "build_id_match"); + } else { + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_IP, "build_id_status"); + } + +cleanup: + test_build_id__destroy(skel); +} + +static void subtest_sleepable(void) +{ + struct test_build_id *skel; + struct bpf_stack_build_id *stack; + int frame_cnt; + + skel = test_build_id__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + skel->links.uprobe_sleepable = bpf_program__attach(skel->progs.uprobe_sleepable); + if (!ASSERT_OK_PTR(skel->links.uprobe_sleepable, "link")) + goto cleanup; + + /* force build ID to not be paged in */ + ASSERT_OK(system("./uprobe_multi uprobe-paged-out"), "trigger_uprobe"); + + if (!ASSERT_GT(skel->bss->res_sleepable, 0, "res")) + goto cleanup; + + stack = skel->bss->stack_sleepable; + frame_cnt = skel->bss->res_sleepable / sizeof(struct bpf_stack_build_id); + if (env.verbosity >= VERBOSE_NORMAL) + print_stack(stack, frame_cnt); + + ASSERT_EQ(stack[0].status, BPF_STACK_BUILD_ID_VALID, "build_id_status"); + ASSERT_EQ(memcmp(stack[0].build_id, build_id, build_id_sz), 0, "build_id_match"); + +cleanup: + test_build_id__destroy(skel); +} + +void test_build_id(void) +{ + build_id_sz = read_build_id("uprobe_multi", build_id, sizeof(build_id)); + ASSERT_EQ(build_id_sz, BPF_BUILD_ID_SIZE, "parse_build_id"); + + if (test__start_subtest("nofault-paged-out")) + subtest_nofault(false /* not resident */); + if (test__start_subtest("nofault-paged-in")) + subtest_nofault(true /* resident */); + if (test__start_subtest("sleepable")) + subtest_sleepable(); +} diff --git a/tools/testing/selftests/bpf/progs/test_build_id.c b/tools/testing/selftests/bpf/progs/test_build_id.c new file mode 100644 index 000000000000..32ce59f9aa27 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_build_id.c @@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include + +struct bpf_stack_build_id stack_sleepable[128]; +int res_sleepable; + +struct bpf_stack_build_id stack_nofault[128]; +int res_nofault; + +SEC("uprobe.multi/./uprobe_multi:uprobe") +int uprobe_nofault(struct pt_regs *ctx) +{ + res_nofault = bpf_get_stack(ctx, stack_nofault, sizeof(stack_nofault), + BPF_F_USER_STACK | BPF_F_USER_BUILD_ID); + + return 0; +} + +SEC("uprobe.multi.s/./uprobe_multi:uprobe") +int uprobe_sleepable(struct pt_regs *ctx) +{ + res_sleepable = bpf_get_stack(ctx, stack_sleepable, sizeof(stack_sleepable), + BPF_F_USER_STACK | BPF_F_USER_BUILD_ID); + + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/uprobe_multi.c b/tools/testing/selftests/bpf/uprobe_multi.c index 7ffa563ffeba..edd7ef3bd5cd 100644 --- a/tools/testing/selftests/bpf/uprobe_multi.c +++ b/tools/testing/selftests/bpf/uprobe_multi.c @@ -2,6 +2,10 @@ #include #include +#include +#include +#include +#include #include #define __PASTE(a, b) a##b @@ -75,6 +79,32 @@ static int usdt(void) return 0; } +int __attribute__((weak)) uprobe(void) +{ + return 0; +} + +extern char build_id_start[]; +extern char build_id_end[]; + +int __attribute__((weak)) trigger_uprobe(bool build_id_resident) +{ + int page_sz = sysconf(_SC_PAGESIZE); + void *addr; + + /* page-align build ID start */ + addr = (void *)((uintptr_t)&build_id_start & ~(page_sz - 1)); + + if (build_id_resident) + madvise(addr, page_sz, MADV_POPULATE_READ); + else + madvise(addr, page_sz, MADV_PAGEOUT); + + (void)uprobe(); + + return 0; +} + int main(int argc, char **argv) { if (argc != 2) @@ -84,6 +114,10 @@ int main(int argc, char **argv) return bench(); if (!strcmp("usdt", argv[1])) return usdt(); + if (!strcmp("uprobe-paged-out", argv[1])) + return trigger_uprobe(false /* page-out build ID */); + if (!strcmp("uprobe-paged-in", argv[1])) + return trigger_uprobe(true /* page-in build ID */); error: fprintf(stderr, "usage: %s \n", argv[0]); diff --git a/tools/testing/selftests/bpf/uprobe_multi.ld b/tools/testing/selftests/bpf/uprobe_multi.ld new file mode 100644 index 000000000000..a2e94828bc8c --- /dev/null +++ b/tools/testing/selftests/bpf/uprobe_multi.ld @@ -0,0 +1,11 @@ +SECTIONS +{ + . = ALIGN(4096); + .note.gnu.build-id : { *(.note.gnu.build-id) } + . = ALIGN(4096); +} +INSERT AFTER .text; + +build_id_start = ADDR(.note.gnu.build-id); +build_id_end = ADDR(.note.gnu.build-id) + SIZEOF(.note.gnu.build-id); +