From patchwork Sat Nov 9 01:28:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 13869137 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDB251C6B8 for ; Sat, 9 Nov 2024 01:28:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115738; cv=none; b=fm9aO7jv4zYHCOctRoWKNHJrhwwFI8B8zVCZhKVu8SqwkLlbzYJl/qE02md9Kch8ksc7nsTphV1owxTPidKSjXLDBrVJd7IkznibCQjLRo/USNvLj1ZjgjWL5ErASZj0n2D3LR62Tz/GY8Sj50R0tcptmCXMUfYc8IQURhMrqww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115738; c=relaxed/simple; bh=U4/39m1wSnM6NEKJmtt/qupJOrq3/D+YkHzzTDlhxIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VN5FtNslE7nzpp6Dw2PZqX28kbSJUDHjlIz2qKwzXgzuNa0CxduOrSLwAiUWGQTloVHQ+uW/AXZloFNFnPxw1dCHEYqpBfd8uceWP3c+w0AmT++HeMP/93UbnRKzWfWKbLIprWPZGbQhOqhRI6KXxR+HuSxMHU7B6UMvjG1Wh74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=GiyvBhg4; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="GiyvBhg4" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-20c9d8563cbso1802295ad.2 for ; Fri, 08 Nov 2024 17:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115736; x=1731720536; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1JQhJjVLe/b9J3jK6ApFcepDaYbpXUd3aMShB4aDox4=; b=GiyvBhg4DWVWTHZf2VGtIgqWYyCAKimbPYxH7LjPXINyntoKf1QIWMv/Dvvg45aKtX QJhk1UrRTsnPk5nzJUrcKMzfhStyl3yoqOhvZiP1wrBKZVV7RXhEAW55N09WA662DYNy M5m7YL0f9oF2ju7fYgKaCMU44N1T0TBezqqwMWPYQtYqOfENfmQck544h66v2p2bOD26 fht5MBIf9/qzauxor3SKhAJAxZlzLDxe7S4ONw3+ctQkmjgvQ3bRtkEEVAufUN/YG4QQ Cg/1dJOqjF0Pv5vqcX0wM0Svyfxv0r8StvPHk/BlALPMRYOyWx1Bbo+Aw8IZxqvUcKRJ uobQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115736; x=1731720536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1JQhJjVLe/b9J3jK6ApFcepDaYbpXUd3aMShB4aDox4=; b=nuSghXH81TAMHdsH+B0IQIxDISAvpRpp4Ly1LU7iwZJAQhX/HjqwiAXQ1h1Z59mTzK BLyPX7o/ViPo6UpzUnGNJ7b0jYUD/bD/cN3BE4rC9RVc3B0VeqOfs4htsz/rVOtpmFZ4 iV02DSIGpv8yc2FhfQGKREI6kvBAu/JPoWJn4lUJ25Pb43nudONPGPyw6gNn8+C9i/Fs KItQ4hTaO6z9IB+n9HTx7ksiArfuDJNtGqmpO3U/wrgzN1WXfdeB/qpeWTee7iVTGT3/ acad1nidCspuFceKQdisSa+YuvGuLjMmiyrgnnC8R6q7HiMcf0jcpTLG3/GdCaya/w0J jZRw== X-Gm-Message-State: AOJu0YzlD9OZ0vRe42p7u2eN1J/LDIkMQX4YUjE/4G5QxTPXDDUEzn+T ydEGUpwPxBwHeU39Qr4Qjm3JEZPhFDD6GsEvtr2hrIhoevywiyS+SXCTK1fvoxX9dBrvh0lolrg L X-Google-Smtp-Source: AGHT+IFKg+q7iegnZHiSf+idYwfgc2LKuVrDKq4p8WHgcGx0frkz7U7cZpi6uSNtCUvntKMxtTgqog== X-Received: by 2002:a17:902:d4cd:b0:20c:da9a:d5b9 with SMTP id d9443c01a7336-211835107c0mr29325685ad.5.1731115735628; Fri, 08 Nov 2024 17:28:55 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:54 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] proc/kcore: mark proc entry as permanent Date: Fri, 8 Nov 2024 17:28:39 -0800 Message-ID: <60873e6afcfda3f08d0456f19e4733612afcf134.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Omar Sandoval drgn reads from /proc/kcore to debug the running kernel. For many drgn scripts, /proc/kcore is actually a bottleneck. use_pde() and unuse_pde() in prog_reg_read() show up hot in profiles. Since the entry for /proc/kcore can never be removed, this is useless overhead that can be trivially avoided by marking the entry as permanent. In my benchmark, this reduces the time per read by about 20 nanoseconds, from 235 nanoseconds per read to 215. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 51446c59388f..770e4e57f445 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -662,6 +662,7 @@ static int release_kcore(struct inode *inode, struct file *file) } static const struct proc_ops kcore_proc_ops = { + .proc_flags = PROC_ENTRY_PERMANENT, .proc_read_iter = read_kcore_iter, .proc_open = open_kcore, .proc_release = release_kcore, From patchwork Sat Nov 9 01:28:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 13869138 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D96F2288B5 for ; Sat, 9 Nov 2024 01:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115739; cv=none; b=Lwa0JUOWjtQU6e7xFPXOrd9P2pfMNjxTBXw3AStvn+TGe4cuzeDiYIzFR/1Hg+1N1gkeRiOCXw9GosdwKb6K0Y+jpHrCB0T2sg8ssyb3EfeS8tNlxoOxrL6Aihn36hl2WnlKXg796uticqttflD/9M/rVrnFKwLwAM9M72Qweq8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115739; c=relaxed/simple; bh=XrcsrSAOKXBBBQwVA9Tuimkn9w8T10KOuIdPu7i4L1Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ISxepNemx3cnVHOxMEwzm5lY2WgaIZnj0FrhVxjiPI66UzJCcxsI5T99608dolLCU7oAXlHilhbfTpnbcFjGly1dfjiv50Ma4pNJB/7IKpdpDcjZGZLZQVvd/P/pDhzLVHOu7TVmqsxzFE5dQodGX6nU8nmFXoX7MbuX33l0f5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=KT3cFvwj; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="KT3cFvwj" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-20c78a10eb3so8125ad.0 for ; Fri, 08 Nov 2024 17:28:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115737; x=1731720537; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bsQOXM5eEzPefp4DDDW0FE1HiiPiLYyB+AE/vJxP/T4=; b=KT3cFvwjStx4TIdQ7YP5oRlY8JjUadpER7nSGZOhd8xcEWdi+KCCgxGgtCUWNXiogP vj2ryIqJKmjJQcyf2Ks6+Ya8DHNpkA0YBn44VDVoGfjRqykpLM/C6eQ01387nAA4xcNu wKGTWx2+BW7KrxKv7l0ZSwKedHfqWzAPafhmtItm9PaR+itNgXaBYjN9vSPCDyqHFhWU SIXCLcBYCTiD8A3gtHp7iL9zijbxGZVD9cnIhAxFF8ueU+Xv4/B1XNi+UR3ndFF7qL2D uYcDkyg5aHaKBgMpu0KE3YLjHwht5TUo4toLbL8pyvGaa0k4FVf/ss0xFrlvDpPKibly hJ5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115737; x=1731720537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bsQOXM5eEzPefp4DDDW0FE1HiiPiLYyB+AE/vJxP/T4=; b=nR85Tq17Yu97mY/Sabg5qgUn8kgZGUS0/cqjVzioAIUnqDBp6b+U8cBp8NdZBkdbH2 2+GrfDSHjWCPjHmnMqh4rZ8XBzrzck/O3fvDjgEcnt2swRWbAxYfj93FwbNYUpyb86Oe Kkjxf4QzdqFbOXgXkhWS59U8fLG43vlnE8xhl+UTO8d9NDmx6qVZdJ5y0seVi5q5a38B /czz52zHodDRMaR25TglEWZ6jjeiF8c8DDdsL/l/CvqlL09H8S7fjeTQyqrLvW8XYWUI MTxToJdUxa+RvTVRAZ7WnbKH5bjAAkIOUASjFAbmKuP2pCqKtcucHPA1Z3Scy0BY4zn+ zM7g== X-Gm-Message-State: AOJu0Yz1JieGSAugjZxIjPn7d/THKOFULot7zC7daaaF8/n/j4jTeDH7 1UhWVw2FzYXSVzBqw10yUC0Bz4xBU5Egxe44vba6u+2mw2Xp96mslSct5RGAd7rrsdurzg/X1Od E X-Google-Smtp-Source: AGHT+IEMcxzKuKijFu4ssmt1kgbjUcwDcCVAU1kIvePW6gaP9G0DE3o/yXMPTKHGS67Lr03Hm8FB/A== X-Received: by 2002:a17:902:ecd0:b0:20c:ee32:7595 with SMTP id d9443c01a7336-211834de6d9mr27903515ad.2.1731115736792; Fri, 08 Nov 2024 17:28:56 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:55 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] proc/kcore: don't walk list on every read Date: Fri, 8 Nov 2024 17:28:40 -0800 Message-ID: <8d945558b9c9efe74103a34b7780f1cd90d9ce7f.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Omar Sandoval We maintain a list of memory ranges for /proc/kcore, which usually has 10-20 entries. Currently, every single read from /proc/kcore walks the entire list in order to count the number of entries and compute some offsets. These values only change when the list of memory ranges changes, which is very rare (only when memory is hot(un)plugged). We can cache the values when the list is populated to avoid these redundant walks. In my benchmark, this reduces the time per read by another 20 nanoseconds on top of the previous change, from 215 nanoseconds per read to 195. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 70 ++++++++++++++++++++++++------------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 770e4e57f445..082718f5c02f 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -65,6 +65,10 @@ static inline void kc_unxlate_dev_mem_ptr(phys_addr_t phys, void *virt) #endif static LIST_HEAD(kclist_head); +static int kcore_nphdr; +static size_t kcore_phdrs_len; +static size_t kcore_notes_len; +static size_t kcore_data_offset; static DECLARE_RWSEM(kclist_lock); static int kcore_need_update = 1; @@ -101,33 +105,32 @@ void __init kclist_add(struct kcore_list *new, void *addr, size_t size, list_add_tail(&new->list, &kclist_head); } -static size_t get_kcore_size(int *nphdr, size_t *phdrs_len, size_t *notes_len, - size_t *data_offset) +static void update_kcore_size(void) { size_t try, size; struct kcore_list *m; - *nphdr = 1; /* PT_NOTE */ + kcore_nphdr = 1; /* PT_NOTE */ size = 0; list_for_each_entry(m, &kclist_head, list) { try = kc_vaddr_to_offset((size_t)m->addr + m->size); if (try > size) size = try; - *nphdr = *nphdr + 1; + kcore_nphdr++; } - *phdrs_len = *nphdr * sizeof(struct elf_phdr); - *notes_len = (4 * sizeof(struct elf_note) + - 3 * ALIGN(sizeof(CORE_STR), 4) + - VMCOREINFO_NOTE_NAME_BYTES + - ALIGN(sizeof(struct elf_prstatus), 4) + - ALIGN(sizeof(struct elf_prpsinfo), 4) + - ALIGN(arch_task_struct_size, 4) + - ALIGN(vmcoreinfo_size, 4)); - *data_offset = PAGE_ALIGN(sizeof(struct elfhdr) + *phdrs_len + - *notes_len); - return *data_offset + size; + kcore_phdrs_len = kcore_nphdr * sizeof(struct elf_phdr); + kcore_notes_len = (4 * sizeof(struct elf_note) + + 3 * ALIGN(sizeof(CORE_STR), 4) + + VMCOREINFO_NOTE_NAME_BYTES + + ALIGN(sizeof(struct elf_prstatus), 4) + + ALIGN(sizeof(struct elf_prpsinfo), 4) + + ALIGN(arch_task_struct_size, 4) + + ALIGN(vmcoreinfo_size, 4)); + kcore_data_offset = PAGE_ALIGN(sizeof(struct elfhdr) + kcore_phdrs_len + + kcore_notes_len); + proc_root_kcore->size = kcore_data_offset + size; } #ifdef CONFIG_HIGHMEM @@ -270,8 +273,6 @@ static int kcore_update_ram(void) { LIST_HEAD(list); LIST_HEAD(garbage); - int nphdr; - size_t phdrs_len, notes_len, data_offset; struct kcore_list *tmp, *pos; int ret = 0; @@ -293,8 +294,7 @@ static int kcore_update_ram(void) } list_splice_tail(&list, &kclist_head); - proc_root_kcore->size = get_kcore_size(&nphdr, &phdrs_len, ¬es_len, - &data_offset); + update_kcore_size(); out: up_write(&kclist_lock); @@ -326,12 +326,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) struct file *file = iocb->ki_filp; char *buf = file->private_data; loff_t *fpos = &iocb->ki_pos; - size_t phdrs_offset, notes_offset, data_offset; + size_t phdrs_offset, notes_offset; size_t page_offline_frozen = 1; - size_t phdrs_len, notes_len; struct kcore_list *m; size_t tsz; - int nphdr; unsigned long start; size_t buflen = iov_iter_count(iter); size_t orig_buflen = buflen; @@ -344,9 +342,8 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) */ page_offline_freeze(); - get_kcore_size(&nphdr, &phdrs_len, ¬es_len, &data_offset); phdrs_offset = sizeof(struct elfhdr); - notes_offset = phdrs_offset + phdrs_len; + notes_offset = phdrs_offset + kcore_phdrs_len; /* ELF file header. */ if (buflen && *fpos < sizeof(struct elfhdr)) { @@ -368,7 +365,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) .e_flags = ELF_CORE_EFLAGS, .e_ehsize = sizeof(struct elfhdr), .e_phentsize = sizeof(struct elf_phdr), - .e_phnum = nphdr, + .e_phnum = kcore_nphdr, }; tsz = min_t(size_t, buflen, sizeof(struct elfhdr) - *fpos); @@ -382,10 +379,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) } /* ELF program headers. */ - if (buflen && *fpos < phdrs_offset + phdrs_len) { + if (buflen && *fpos < phdrs_offset + kcore_phdrs_len) { struct elf_phdr *phdrs, *phdr; - phdrs = kzalloc(phdrs_len, GFP_KERNEL); + phdrs = kzalloc(kcore_phdrs_len, GFP_KERNEL); if (!phdrs) { ret = -ENOMEM; goto out; @@ -393,13 +390,14 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) phdrs[0].p_type = PT_NOTE; phdrs[0].p_offset = notes_offset; - phdrs[0].p_filesz = notes_len; + phdrs[0].p_filesz = kcore_notes_len; phdr = &phdrs[1]; list_for_each_entry(m, &kclist_head, list) { phdr->p_type = PT_LOAD; phdr->p_flags = PF_R | PF_W | PF_X; - phdr->p_offset = kc_vaddr_to_offset(m->addr) + data_offset; + phdr->p_offset = kc_vaddr_to_offset(m->addr) + + kcore_data_offset; phdr->p_vaddr = (size_t)m->addr; if (m->type == KCORE_RAM) phdr->p_paddr = __pa(m->addr); @@ -412,7 +410,8 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) phdr++; } - tsz = min_t(size_t, buflen, phdrs_offset + phdrs_len - *fpos); + tsz = min_t(size_t, buflen, + phdrs_offset + kcore_phdrs_len - *fpos); if (copy_to_iter((char *)phdrs + *fpos - phdrs_offset, tsz, iter) != tsz) { kfree(phdrs); @@ -426,7 +425,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) } /* ELF note segment. */ - if (buflen && *fpos < notes_offset + notes_len) { + if (buflen && *fpos < notes_offset + kcore_notes_len) { struct elf_prstatus prstatus = {}; struct elf_prpsinfo prpsinfo = { .pr_sname = 'R', @@ -438,7 +437,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) strscpy(prpsinfo.pr_psargs, saved_command_line, sizeof(prpsinfo.pr_psargs)); - notes = kzalloc(notes_len, GFP_KERNEL); + notes = kzalloc(kcore_notes_len, GFP_KERNEL); if (!notes) { ret = -ENOMEM; goto out; @@ -459,9 +458,10 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) */ append_kcore_note(notes, &i, VMCOREINFO_NOTE_NAME, 0, vmcoreinfo_data, - min(vmcoreinfo_size, notes_len - i)); + min(vmcoreinfo_size, kcore_notes_len - i)); - tsz = min_t(size_t, buflen, notes_offset + notes_len - *fpos); + tsz = min_t(size_t, buflen, + notes_offset + kcore_notes_len - *fpos); if (copy_to_iter(notes + *fpos - notes_offset, tsz, iter) != tsz) { kfree(notes); ret = -EFAULT; @@ -477,7 +477,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) * Check to see if our file offset matches with any of * the addresses in the elf_phdr on our list. */ - start = kc_offset_to_vaddr(*fpos - data_offset); + start = kc_offset_to_vaddr(*fpos - kcore_data_offset); if ((tsz = (PAGE_SIZE - (start & ~PAGE_MASK))) > buflen) tsz = buflen; From patchwork Sat Nov 9 01:28:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 13869139 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDD9645025 for ; Sat, 9 Nov 2024 01:28:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115740; cv=none; b=S5f45OIKyZXhgK4qBZYgdtD44guCfvgBDQ/WuUGPbAXGndrmWgQ0GrMVqLMdlwdEKz68MFBP4Xuol2L7k1490MCX/6EC8knXSk4FyhwLFbezh9bkkwlGp5wzy2AzVsTUGhEZvIqIZCqAS5JJvkvhtZDvHjp1L6y3AjrVoPwSUpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115740; c=relaxed/simple; bh=2V+LMyhP7QCx7vipIrK/PNDo/NUYimbVbH9Zga/QRnw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lGNnLONdqzDyQKZszGSJjfHz7Z2PEUtI5M1S0bsCvfSqpQNVHpfSlE5wh+mqTP/LA8IyzMyRu02MRhB6SxLFfX5Zpvxr8QMV9pB/hW9WtJ27LfQamQkyzzKXlUgFhmYNCBAkqOywqsQPPAsmt8oTThAikHmZlYwV63cqWrMb00A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=SJM1fV2i; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="SJM1fV2i" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2e2c2a17aa4so325302a91.0 for ; Fri, 08 Nov 2024 17:28:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115738; x=1731720538; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xOqgPPtziEx8SGLaOYb14oaOw/i0JcmNn0bH4qe21tg=; b=SJM1fV2i8umSzCb58koP7j7K+NELgsaCQQQH/MkptEuEx7pohw8Zvrn2gBl5q+9kTN 740gtHf9rrk60l0atKcx41Hw9dEeaOoTz3lTLcdJEA8JCmrh9he0hQ5P62EvXd3IM0Lm U2oKGJFSbo6qHB6N3d1UZzZK+VgnjvOtpRTuqAdvwSoZvEyGaXzytFDCeh682XHxCNsq qgBWC3nPkRI6z6DkDwZyNDrUxyalAuw3qwC0xDRHlVt8F+LhnxPJ6g4G/gcolcMJeGTH /WB74lgTewunqEfl7neoRJTCejvuudZRffnm/JTSoLDOxdD8L6nLlKLaFD3INJOW0BaW rEmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115738; x=1731720538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xOqgPPtziEx8SGLaOYb14oaOw/i0JcmNn0bH4qe21tg=; b=t3REvCfhmbYAgPmqouLnyC1XLb/AljZ3dU3nCTYGW6yPyLy9uZQ3Ym7+qLZAddA7Hu F8uv2stYAuJ9PjtVPyIGWYEJe1lFo02dBPDN49TdlBZMDvL7Q4C1U1iQgSs2P5GPhPtt uVnj5YWKZZgN9o22GuFKko2RgsrYKkfc29Kq8B0Z6IUVqqGqlanasrtxL2jHnXGg3LCt Unn3ZRb7uhP7eVWlea4ZDedtlfwPrBkIIdnmPCkJDR3IGG5wjw5qNaZdyAqFKeQoBU8R B8OSTKZPaOurT2vCqSnU2j0cvBDyM1rOnf9DDfBHgQDdqENDOed3YWzSrHrATnDseIL/ aukQ== X-Gm-Message-State: AOJu0YxOph6srr0d1T4eWcREZsv5lC8rdm9+6mIQ95JUSDmNxl12s12I pHPfZG7m+zgXZw+gHNBY36Z60aRrPL4svVpJnxPXFdSYuwAzTpwIJtCyQ7U0UICBgQJxJXrjBJC J X-Google-Smtp-Source: AGHT+IFOd7MaBqO9qvnNSmqC8iCKDohmW0xsMff0WgR9ONvpPAG2Els+t+TmPhLbjIw8o7WypoTk7g== X-Received: by 2002:a17:902:f68a:b0:20c:e169:eb6a with SMTP id d9443c01a7336-2118357e1b5mr27424505ad.10.1731115737917; Fri, 08 Nov 2024 17:28:57 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:57 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] proc/kcore: use percpu_rw_semaphore for kclist_lock Date: Fri, 8 Nov 2024 17:28:41 -0800 Message-ID: <83a3b235b4bcc3b8aef7c533e0657f4d7d5d35ae.1731115587.git.osandov@fb.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Omar Sandoval The list of memory ranges for /proc/kcore is protected by a rw_semaphore. We lock it for reading on every read from /proc/kcore. This is very heavy, especially since it is rarely locked for writing. Since we want to strongly favor read lock performance, convert it to a percpu_rw_semaphore. I also experimented with percpu_ref and SRCU, but this change was the simplest and the fastest. In my benchmark, this reduces the time per read by yet another 20 nanoseconds on top of the previous two changes, from 195 nanoseconds per read to 175. Link: https://github.com/osandov/drgn/issues/106 Signed-off-by: Omar Sandoval --- fs/proc/kcore.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index 082718f5c02f..f0d56d000816 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -69,7 +69,7 @@ static int kcore_nphdr; static size_t kcore_phdrs_len; static size_t kcore_notes_len; static size_t kcore_data_offset; -static DECLARE_RWSEM(kclist_lock); +DEFINE_STATIC_PERCPU_RWSEM(kclist_lock); static int kcore_need_update = 1; /* @@ -276,7 +276,7 @@ static int kcore_update_ram(void) struct kcore_list *tmp, *pos; int ret = 0; - down_write(&kclist_lock); + percpu_down_write(&kclist_lock); if (!xchg(&kcore_need_update, 0)) goto out; @@ -297,7 +297,7 @@ static int kcore_update_ram(void) update_kcore_size(); out: - up_write(&kclist_lock); + percpu_up_write(&kclist_lock); list_for_each_entry_safe(pos, tmp, &garbage, list) { list_del(&pos->list); kfree(pos); @@ -335,7 +335,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) size_t orig_buflen = buflen; int ret = 0; - down_read(&kclist_lock); + percpu_down_read(&kclist_lock); /* * Don't race against drivers that set PageOffline() and expect no * further page access. @@ -625,7 +625,7 @@ static ssize_t read_kcore_iter(struct kiocb *iocb, struct iov_iter *iter) out: page_offline_thaw(); - up_read(&kclist_lock); + percpu_up_read(&kclist_lock); if (ret) return ret; return orig_buflen - buflen; From patchwork Sat Nov 9 01:28:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Omar Sandoval X-Patchwork-Id: 13869140 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D34A5558A5 for ; Sat, 9 Nov 2024 01:28:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115741; cv=none; b=pE2mpny7xaVon6OEaEHVv1CWk/+86ws3O1zNtJQU6XHlqdSfbDEteRH4fpMp5D20D9RsvVti/Qx2TwmM8Lxen5vnhV1lfNj8HrqCMXxTgVSbQTKFZ05XGQI+mm9hXkaxQVieiZy7SVJu0yhoHJpkhLp4oOgdOvLg9vypl9JiVKw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731115741; c=relaxed/simple; bh=uZOIUaRs+7H+jn62BpsP+uX28ASu2b/m0Dd0veYqTPs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BAxSZZJrRwdA4UMaoH7fz3uCHL5mNbi10DcilgsFdtw3xVNinOFOsq7LNV3nDa232bTKh5ZkuN5Ay2hl0Cf7M2n11vB7EBQq+cSPuvHdNWibzJFT8I6YNkIVQsUllV4W5UJVHOjlWko78XZ9L0U743y3qhitXc5ngv+RrHpVKPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com; spf=none smtp.mailfrom=osandov.com; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b=uWa5rgwV; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=osandov.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=osandov.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=osandov-com.20230601.gappssmtp.com header.i=@osandov-com.20230601.gappssmtp.com header.b="uWa5rgwV" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-20c7b9087c4so6395ad.1 for ; Fri, 08 Nov 2024 17:28:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osandov-com.20230601.gappssmtp.com; s=20230601; t=1731115739; x=1731720539; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e4yNjGAg8qf3fKwWurXSx/s8j5dgkc97k/3nCeXjiZI=; b=uWa5rgwV4reeQFAdaDLk35MfwdFUPJ+wVO6Da+WNWAYyb5XPToSp87v5GfTcvvf+Iw YFNI/j6fwJbqa/71fHRWmClD5tPNynamdv5tsaHarZ0XVm/3hAReAFLb+pmxJgwDH4Nr dhHT6U/ex2MYmEiI7rpFj5u51hD3Eh0HAmk0iTB9bToJpH/KHfo80EW/YX/GEa90LTmE LbFZr9j6kSl4aF0y0ztLQ29KXgVP0wI1FabeW4z3P/5H+ILw1bqFD7Fuqdi4fma8WfcV +TeTaxTHhC3PmAWqA95HbMzaX/LPjfsUFpXzvt1KEq0z/F4JSjKUydICAKw/PssWD9qN eqrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731115739; x=1731720539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e4yNjGAg8qf3fKwWurXSx/s8j5dgkc97k/3nCeXjiZI=; b=dYAmqnpXKF1jv37P+VRf0/YgODk6z/iltpN99PmYYawEWDgmI7R6qSDrtbpbikLB6e saDwTvwNlOOHm9rdCaYRjyrqUvmZz3Ds2qQYoBgeUOy6FcmkfNFe5rvPq7qsU9eSCPId 7tf+fsswZwAByWfEyeIDbHyze6fbuicHweCUyOPkswBzIqRRqKMH+HgHT29YOB/IsvpQ DWMXg4FJQrheOg9m6bucZJbTFgDav7bs0MZTEaBfhJV42XpsyHR3JioIzxn/Cn0rPj2a Ku9Zn2zZa/OH74/epO618QNW2oY5VW2n+cVPA8DYZGVzFkVnoBHeJRnFt/MSlbvpX+FF fKBQ== X-Gm-Message-State: AOJu0YwrHydmfnVF/HF3wcwbO+NxSjKCH0O1qMnTMqPmsK4QD5g7t1Ld kluueh5mdwzW378Lfc6FhGeksN1MJ6+0+b5wDUPhjmQ4TkCV0vkJ6tdntoIuyQliyL2GUebTDwL l X-Google-Smtp-Source: AGHT+IGxDPbGoufHSHqpkSKM+XO+452K3765m0FRuWx/5mfOxgW3YLlOJkt1wHWgGfUa1caBsRR/ng== X-Received: by 2002:a17:902:dac9:b0:20b:80e6:bce6 with SMTP id d9443c01a7336-211834f8c81mr28605025ad.4.1731115738993; Fri, 08 Nov 2024 17:28:58 -0800 (PST) Received: from telecaster.hsd1.wa.comcast.net ([2601:602:8980:9170::5633]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21177e6c96fsm37493355ad.255.2024.11.08.17.28.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 17:28:58 -0800 (PST) From: Omar Sandoval To: linux-fsdevel@vger.kernel.org, Al Viro Cc: kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] MAINTAINERS: add me as /proc/kcore maintainer Date: Fri, 8 Nov 2024 17:28:42 -0800 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Omar Sandoval Christian volunteered me for this a while back given that drgn is the main user of /proc/kcore and I've touched it several times over the years. Link: https://lore.kernel.org/all/20231125-kurhotel-zuwege-10cce62a50fd@brauner/ Signed-off-by: Omar Sandoval --- MAINTAINERS | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index bdae0faf000c..89645de6faba 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12242,6 +12242,13 @@ F: Documentation/kbuild/kconfig* F: scripts/Kconfig.include F: scripts/kconfig/ +KCORE +M: Omar Sandoval +L: linux-debuggers@vger.kernel.org +S: Maintained +F: fs/proc/kcore.c +F: include/linux/kcore.h + KCOV R: Dmitry Vyukov R: Andrey Konovalov