From patchwork Wed Feb 23 19:38:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 12757455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B3EC433FE for ; Wed, 23 Feb 2022 19:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244471AbiBWTih (ORCPT ); Wed, 23 Feb 2022 14:38:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233923AbiBWTig (ORCPT ); Wed, 23 Feb 2022 14:38:36 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFFB348E6A for ; Wed, 23 Feb 2022 11:38:08 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 803371F37E; Wed, 23 Feb 2022 19:38:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1645645087; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s5ZN7uI6UG6Di2iSINpGQFR90gjSNoFjkDK2XKHJmb8=; b=PCa6oGDrQ1n2qcRHF5a73W6x6YgnrbQsuD32eUczTD2snckw0qWUf22KsXT0Pv7dQ+Jhaa CE7FslEwt4wDJywN4UCFyNds61jSLj7sjcsSrnCX+ECjRe0izJewbRoHLsrspAW1pupja9 5XnM9sJTjDmPeWP8YwYmK2Tl6/D5oKo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1645645087; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s5ZN7uI6UG6Di2iSINpGQFR90gjSNoFjkDK2XKHJmb8=; b=Jx64EzN+aniD/z/Ga2FsNuAbxSPpCLCdhHw+MhW2Vu5gLLuGu9Va55v559ikUYFAK9dQ/t hSlW0/dUP53WjDCw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 47FD513DAC; Wed, 23 Feb 2022 19:38:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id WOaNEB+NFmKfUgAAMHmgww (envelope-from ); Wed, 23 Feb 2022 19:38:07 +0000 From: Thomas Zimmermann To: daniel@ffwll.ch, deller@gmx.de, javierm@redhat.com, geert@linux-m68k.org, sam@ravnborg.org, kraxel@redhat.com, ppaalanen@gmail.com Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, Thomas Zimmermann Subject: [PATCH v3 1/5] fbdev: Improve performance of sys_fillrect() Date: Wed, 23 Feb 2022 20:38:00 +0100 Message-Id: <20220223193804.18636-2-tzimmermann@suse.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220223193804.18636-1-tzimmermann@suse.de> References: <20220223193804.18636-1-tzimmermann@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Improve the performance of sys_fillrect() by using word-aligned 32/64-bit mov instructions. While the code tried to implement this, the compiler failed to create fast instructions. The resulting binary instructions were even slower than cfb_fillrect(), which uses the same algorithm, but operates on I/O memory. A microbenchmark measures the average number of CPU cycles for sys_fillrect() after a stabilizing period of a few minutes (i7-4790, FullHD, simpledrm, kernel with debugging). The value for CFB is given as a reference. sys_fillrect(), new: 26586 cycles sys_fillrect(), old: 166603 cycles cfb_fillrect(): 41012 cycles In the optimized case, sys_fillrect() is now ~6x faster than before and ~1.5x faster than the CFB implementation. Signed-off-by: Thomas Zimmermann Reviewed-by: Javier Martinez Canillas Reviewed-by: Sam Ravnborg --- drivers/video/fbdev/core/sysfillrect.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/drivers/video/fbdev/core/sysfillrect.c b/drivers/video/fbdev/core/sysfillrect.c index 33ee3d34f9d2..bcdcaeae6538 100644 --- a/drivers/video/fbdev/core/sysfillrect.c +++ b/drivers/video/fbdev/core/sysfillrect.c @@ -50,19 +50,9 @@ bitfill_aligned(struct fb_info *p, unsigned long *dst, int dst_idx, /* Main chunk */ n /= bits; - while (n >= 8) { - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - *dst++ = pat; - n -= 8; - } - while (n--) - *dst++ = pat; + memset_l(dst, pat, n); + dst += n; + /* Trailing bits */ if (last) *dst = comp(pat, *dst, last); From patchwork Wed Feb 23 19:38:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 12757457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0F0DC4332F for ; Wed, 23 Feb 2022 19:38:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244474AbiBWTih (ORCPT ); Wed, 23 Feb 2022 14:38:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238574AbiBWTih (ORCPT ); Wed, 23 Feb 2022 14:38:37 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FFB848E6B for ; Wed, 23 Feb 2022 11:38:09 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BD17B1F44C; Wed, 23 Feb 2022 19:38:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1645645087; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N+GbnOGSeR3X9TVxFGABxV5nEPCZ3RUvOlA//TY0ScI=; b=nYz7FNsAjd1IIOARUyXyPa+7heUpaOGS7DQ1SFaIasN1VX3DQ6ROtnFIZpLgqzqSGvzags TRPaT8RN7BzyJzxvRVlTp+/+e/msZPGWd9eu8gbCnmv+AuYigrdJuZPzn3zKTC4vdalN9Q j1/rHR99mURusjp2N95xv6UH3GbIrFI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1645645087; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N+GbnOGSeR3X9TVxFGABxV5nEPCZ3RUvOlA//TY0ScI=; b=cpJfvzZ3AXcc30GzjxW9hlXVk6MDPiAK8GhocIRZuGfWMMdauSjPcTYW/J22NbOIxIkF5b YAZy/a0eI1UNSbBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 856FF13C7B; Wed, 23 Feb 2022 19:38:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id IDODHx+NFmKfUgAAMHmgww (envelope-from ); Wed, 23 Feb 2022 19:38:07 +0000 From: Thomas Zimmermann To: daniel@ffwll.ch, deller@gmx.de, javierm@redhat.com, geert@linux-m68k.org, sam@ravnborg.org, kraxel@redhat.com, ppaalanen@gmail.com Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, Thomas Zimmermann Subject: [PATCH v3 2/5] fbdev: Improve performance of sys_imageblit() Date: Wed, 23 Feb 2022 20:38:01 +0100 Message-Id: <20220223193804.18636-3-tzimmermann@suse.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220223193804.18636-1-tzimmermann@suse.de> References: <20220223193804.18636-1-tzimmermann@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Improve the performance of sys_imageblit() by manually unrolling the inner blitting loop and moving some invariants out. The compiler failed to do this automatically. The resulting binary code was even slower than the cfb_imageblit() helper, which uses the same algorithm, but operates on I/O memory. A microbenchmark measures the average number of CPU cycles for sys_imageblit() after a stabilizing period of a few minutes (i7-4790, FullHD, simpledrm, kernel with debugging). The value for CFB is given as a reference. sys_imageblit(), new: 25934 cycles sys_imageblit(), old: 35944 cycles cfb_imageblit(): 30566 cycles In the optimized case, sys_imageblit() is now ~30% faster than before and ~20% faster than cfb_imageblit(). v2: * move switch out of inner loop (Gerd) * remove test for alignment of dst1 (Sam) Signed-off-by: Thomas Zimmermann Reviewed-by: Javier Martinez Canillas Acked-by: Sam Ravnborg --- drivers/video/fbdev/core/sysimgblt.c | 49 +++++++++++++++++++++------- 1 file changed, 38 insertions(+), 11 deletions(-) diff --git a/drivers/video/fbdev/core/sysimgblt.c b/drivers/video/fbdev/core/sysimgblt.c index a4d05b1b17d7..722c327a381b 100644 --- a/drivers/video/fbdev/core/sysimgblt.c +++ b/drivers/video/fbdev/core/sysimgblt.c @@ -188,23 +188,29 @@ static void fast_imageblit(const struct fb_image *image, struct fb_info *p, { u32 fgx = fgcolor, bgx = bgcolor, bpp = p->var.bits_per_pixel; u32 ppw = 32/bpp, spitch = (image->width + 7)/8; - u32 bit_mask, end_mask, eorx, shift; + u32 bit_mask, eorx; const char *s = image->data, *src; u32 *dst; - const u32 *tab = NULL; + const u32 *tab; + size_t tablen; + u32 colortab[16]; int i, j, k; switch (bpp) { case 8: tab = fb_be_math(p) ? cfb_tab8_be : cfb_tab8_le; + tablen = 16; break; case 16: tab = fb_be_math(p) ? cfb_tab16_be : cfb_tab16_le; + tablen = 4; break; case 32: - default: tab = cfb_tab32; + tablen = 2; break; + default: + return; } for (i = ppw-1; i--; ) { @@ -218,19 +224,40 @@ static void fast_imageblit(const struct fb_image *image, struct fb_info *p, eorx = fgx ^ bgx; k = image->width/ppw; + for (i = 0; i < tablen; ++i) + colortab[i] = (tab[i] & eorx) ^ bgx; + for (i = image->height; i--; ) { dst = dst1; - shift = 8; src = s; - for (j = k; j--; ) { - shift -= ppw; - end_mask = tab[(*src >> shift) & bit_mask]; - *dst++ = (end_mask & eorx) ^ bgx; - if (!shift) { - shift = 8; - src++; + switch (ppw) { + case 4: /* 8 bpp */ + for (j = k; j; j -= 2, ++src) { + *dst++ = colortab[(*src >> 4) & bit_mask]; + *dst++ = colortab[(*src >> 0) & bit_mask]; + } + break; + case 2: /* 16 bpp */ + for (j = k; j; j -= 4, ++src) { + *dst++ = colortab[(*src >> 6) & bit_mask]; + *dst++ = colortab[(*src >> 4) & bit_mask]; + *dst++ = colortab[(*src >> 2) & bit_mask]; + *dst++ = colortab[(*src >> 0) & bit_mask]; + } + break; + case 1: /* 32 bpp */ + for (j = k; j; j -= 8, ++src) { + *dst++ = colortab[(*src >> 7) & bit_mask]; + *dst++ = colortab[(*src >> 6) & bit_mask]; + *dst++ = colortab[(*src >> 5) & bit_mask]; + *dst++ = colortab[(*src >> 4) & bit_mask]; + *dst++ = colortab[(*src >> 3) & bit_mask]; + *dst++ = colortab[(*src >> 2) & bit_mask]; + *dst++ = colortab[(*src >> 1) & bit_mask]; + *dst++ = colortab[(*src >> 0) & bit_mask]; } + break; } dst1 += p->fix.line_length; s += spitch; From patchwork Wed Feb 23 19:38:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 12757458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DBADC4321E for ; Wed, 23 Feb 2022 19:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244477AbiBWTij (ORCPT ); Wed, 23 Feb 2022 14:38:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233923AbiBWTih (ORCPT ); Wed, 23 Feb 2022 14:38:37 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7060D48E7A for ; Wed, 23 Feb 2022 11:38:09 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0593D1F44D; Wed, 23 Feb 2022 19:38:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qObtglYGE46QC6Xcws5G2iEc2tNkficaQIiMH33Sug4=; b=AUsRmDmcZO/dBC7HzVPOOdTSmqSsAZYJULwlCgeTe74hYJLAUUabiPk1bd7j9OnX6MEydI c+Q8HSyQgs+PoLrxO9f/bh2T5+y2v/44/aHnHHquVdp+AGPODh6ZJvWtbUnwx6z7uhFaM4 sm+sBH1c6WMgntx8+JUSpc2bRlO4C/o= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qObtglYGE46QC6Xcws5G2iEc2tNkficaQIiMH33Sug4=; b=kCqxRcpJa8NtmH85C/+IGTeNRLCsmdEWnOW8P07h1USasRu5JAlx+1gVsQsLlc79WnPgJm eXB9cjdgY74dJ1Bw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id C1DB413C7B; Wed, 23 Feb 2022 19:38:07 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yLBOLh+NFmKfUgAAMHmgww (envelope-from ); Wed, 23 Feb 2022 19:38:07 +0000 From: Thomas Zimmermann To: daniel@ffwll.ch, deller@gmx.de, javierm@redhat.com, geert@linux-m68k.org, sam@ravnborg.org, kraxel@redhat.com, ppaalanen@gmail.com Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, Thomas Zimmermann Subject: [PATCH v3 3/5] fbdev: Remove trailing whitespaces from cfbimgblt.c Date: Wed, 23 Feb 2022 20:38:02 +0100 Message-Id: <20220223193804.18636-4-tzimmermann@suse.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220223193804.18636-1-tzimmermann@suse.de> References: <20220223193804.18636-1-tzimmermann@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Fix coding style. No functional changes. Signed-off-by: Thomas Zimmermann Acked-by: Sam Ravnborg Reviewed-by: Javier Martinez Canillas --- drivers/video/fbdev/core/cfbimgblt.c | 60 ++++++++++++++-------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/drivers/video/fbdev/core/cfbimgblt.c b/drivers/video/fbdev/core/cfbimgblt.c index a2bb276a8b24..01b01a279681 100644 --- a/drivers/video/fbdev/core/cfbimgblt.c +++ b/drivers/video/fbdev/core/cfbimgblt.c @@ -16,15 +16,15 @@ * must be laid out exactly in the same format as the framebuffer. Yes I know * their are cards with hardware that coverts images of various depths to the * framebuffer depth. But not every card has this. All images must be rounded - * up to the nearest byte. For example a bitmap 12 bits wide must be two - * bytes width. + * up to the nearest byte. For example a bitmap 12 bits wide must be two + * bytes width. * - * Tony: - * Incorporate mask tables similar to fbcon-cfb*.c in 2.4 API. This speeds + * Tony: + * Incorporate mask tables similar to fbcon-cfb*.c in 2.4 API. This speeds * up the code significantly. - * + * * Code for depths not multiples of BITS_PER_LONG is still kludgy, which is - * still processed a bit at a time. + * still processed a bit at a time. * * Also need to add code to deal with cards endians that are different than * the native cpu endians. I also need to deal with MSB position in the word. @@ -72,8 +72,8 @@ static const u32 cfb_tab32[] = { #define FB_WRITEL fb_writel #define FB_READL fb_readl -static inline void color_imageblit(const struct fb_image *image, - struct fb_info *p, u8 __iomem *dst1, +static inline void color_imageblit(const struct fb_image *image, + struct fb_info *p, u8 __iomem *dst1, u32 start_index, u32 pitch_index) { @@ -92,7 +92,7 @@ static inline void color_imageblit(const struct fb_image *image, dst = (u32 __iomem *) dst1; shift = 0; val = 0; - + if (start_index) { u32 start_mask = ~fb_shifted_pixels_mask_u32(p, start_index, bswapmask); @@ -109,8 +109,8 @@ static inline void color_imageblit(const struct fb_image *image, val |= FB_SHIFT_HIGH(p, color, shift ^ bswapmask); if (shift >= null_bits) { FB_WRITEL(val, dst++); - - val = (shift == null_bits) ? 0 : + + val = (shift == null_bits) ? 0 : FB_SHIFT_LOW(p, color, 32 - shift); } shift += bpp; @@ -134,9 +134,9 @@ static inline void color_imageblit(const struct fb_image *image, } } -static inline void slow_imageblit(const struct fb_image *image, struct fb_info *p, +static inline void slow_imageblit(const struct fb_image *image, struct fb_info *p, u8 __iomem *dst1, u32 fgcolor, - u32 bgcolor, + u32 bgcolor, u32 start_index, u32 pitch_index) { @@ -172,7 +172,7 @@ static inline void slow_imageblit(const struct fb_image *image, struct fb_info * l--; color = (*s & (1 << l)) ? fgcolor : bgcolor; val |= FB_SHIFT_HIGH(p, color, shift ^ bswapmask); - + /* Did the bitshift spill bits to the next long? */ if (shift >= null_bits) { FB_WRITEL(val, dst++); @@ -191,16 +191,16 @@ static inline void slow_imageblit(const struct fb_image *image, struct fb_info * FB_WRITEL((FB_READL(dst) & end_mask) | val, dst); } - + dst1 += pitch; - src += spitch; + src += spitch; if (pitch_index) { dst2 += pitch; dst1 = (u8 __iomem *)((long __force)dst2 & ~(sizeof(u32) - 1)); start_index += pitch_index; start_index &= 32 - 1; } - + } } @@ -212,9 +212,9 @@ static inline void slow_imageblit(const struct fb_image *image, struct fb_info * * fix->line_legth is divisible by 4; * beginning and end of a scanline is dword aligned */ -static inline void fast_imageblit(const struct fb_image *image, struct fb_info *p, - u8 __iomem *dst1, u32 fgcolor, - u32 bgcolor) +static inline void fast_imageblit(const struct fb_image *image, struct fb_info *p, + u8 __iomem *dst1, u32 fgcolor, + u32 bgcolor) { u32 fgx = fgcolor, bgx = bgcolor, bpp = p->var.bits_per_pixel; u32 ppw = 32/bpp, spitch = (image->width + 7)/8; @@ -243,25 +243,25 @@ static inline void fast_imageblit(const struct fb_image *image, struct fb_info * fgx |= fgcolor; bgx |= bgcolor; } - + bit_mask = (1 << ppw) - 1; eorx = fgx ^ bgx; k = image->width/ppw; for (i = image->height; i--; ) { dst = (u32 __iomem *) dst1, shift = 8; src = s; - + for (j = k; j--; ) { shift -= ppw; end_mask = tab[(*src >> shift) & bit_mask]; FB_WRITEL((end_mask & eorx)^bgx, dst++); - if (!shift) { shift = 8; src++; } + if (!shift) { shift = 8; src++; } } dst1 += p->fix.line_length; s += spitch; } -} - +} + void cfb_imageblit(struct fb_info *p, const struct fb_image *image) { u32 fgcolor, bgcolor, start_index, bitstart, pitch_index = 0; @@ -292,13 +292,13 @@ void cfb_imageblit(struct fb_info *p, const struct fb_image *image) } else { fgcolor = image->fg_color; bgcolor = image->bg_color; - } - - if (32 % bpp == 0 && !start_index && !pitch_index && + } + + if (32 % bpp == 0 && !start_index && !pitch_index && ((width & (32/bpp-1)) == 0) && - bpp >= 8 && bpp <= 32) + bpp >= 8 && bpp <= 32) fast_imageblit(image, p, dst1, fgcolor, bgcolor); - else + else slow_imageblit(image, p, dst1, fgcolor, bgcolor, start_index, pitch_index); } else From patchwork Wed Feb 23 19:38:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 12757456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40248C43217 for ; Wed, 23 Feb 2022 19:38:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244476AbiBWTii (ORCPT ); Wed, 23 Feb 2022 14:38:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244472AbiBWTih (ORCPT ); Wed, 23 Feb 2022 14:38:37 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD34748E66 for ; Wed, 23 Feb 2022 11:38:09 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 451F7218A4; Wed, 23 Feb 2022 19:38:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0WC5A0cwjMwFCC8sIvWa7TyxgMdpck0i2lJMuQp2tx4=; b=iof2/chYLLtjdipF/jVkAP2RRBdYGhVIAmUSTRg+UWM5W9yLsrCe5vx45+/tpepso4aerj bGuhuNbjs3ojSj/pLWza3ECFAILFxDzeHP5kvXYaHsIgHJDK0APSIJ6kGzO71Upr2Q6FXa q19jlxcbBrb8WbHaIvgfs8NdA0WM7y0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0WC5A0cwjMwFCC8sIvWa7TyxgMdpck0i2lJMuQp2tx4=; b=IasVD9zwNEO4HfhyxO70oFrt+9ftjSCEkOL0sJWcFWXfJnrp/6wy3b2xqcpxOY0n/EVe28 b3TQdLcNt+ncGbBw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0B40813C7B; Wed, 23 Feb 2022 19:38:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CGDIASCNFmKfUgAAMHmgww (envelope-from ); Wed, 23 Feb 2022 19:38:08 +0000 From: Thomas Zimmermann To: daniel@ffwll.ch, deller@gmx.de, javierm@redhat.com, geert@linux-m68k.org, sam@ravnborg.org, kraxel@redhat.com, ppaalanen@gmail.com Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, Thomas Zimmermann Subject: [PATCH v3 4/5] fbdev: Improve performance of cfb_imageblit() Date: Wed, 23 Feb 2022 20:38:03 +0100 Message-Id: <20220223193804.18636-5-tzimmermann@suse.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220223193804.18636-1-tzimmermann@suse.de> References: <20220223193804.18636-1-tzimmermann@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Improve the performance of cfb_imageblit() by manually unrolling the inner blitting loop and moving some invariants out. The compiler failed to do this automatically. This change keeps cfb_imageblit() in sync with sys_imagebit(). A microbenchmark measures the average number of CPU cycles for cfb_imageblit() after a stabilizing period of a few minutes (i7-4790, FullHD, simpledrm, kernel with debugging). cfb_imageblit(), new: 15724 cycles cfb_imageblit(): old: 30566 cycles In the optimized case, cfb_imageblit() is now ~2x faster than before. v3: * fix commit description (Pekka) Signed-off-by: Thomas Zimmermann Acked-by: Sam Ravnborg Reviewed-by: Javier Martinez Canillas --- drivers/video/fbdev/core/cfbimgblt.c | 51 +++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/drivers/video/fbdev/core/cfbimgblt.c b/drivers/video/fbdev/core/cfbimgblt.c index 01b01a279681..7361cfabdd85 100644 --- a/drivers/video/fbdev/core/cfbimgblt.c +++ b/drivers/video/fbdev/core/cfbimgblt.c @@ -218,23 +218,29 @@ static inline void fast_imageblit(const struct fb_image *image, struct fb_info * { u32 fgx = fgcolor, bgx = bgcolor, bpp = p->var.bits_per_pixel; u32 ppw = 32/bpp, spitch = (image->width + 7)/8; - u32 bit_mask, end_mask, eorx, shift; + u32 bit_mask, eorx; const char *s = image->data, *src; u32 __iomem *dst; const u32 *tab = NULL; + size_t tablen; + u32 colortab[16]; int i, j, k; switch (bpp) { case 8: tab = fb_be_math(p) ? cfb_tab8_be : cfb_tab8_le; + tablen = 16; break; case 16: tab = fb_be_math(p) ? cfb_tab16_be : cfb_tab16_le; + tablen = 4; break; case 32: - default: tab = cfb_tab32; + tablen = 2; break; + default: + return; } for (i = ppw-1; i--; ) { @@ -248,15 +254,42 @@ static inline void fast_imageblit(const struct fb_image *image, struct fb_info * eorx = fgx ^ bgx; k = image->width/ppw; - for (i = image->height; i--; ) { - dst = (u32 __iomem *) dst1, shift = 8; src = s; + for (i = 0; i < tablen; ++i) + colortab[i] = (tab[i] & eorx) ^ bgx; - for (j = k; j--; ) { - shift -= ppw; - end_mask = tab[(*src >> shift) & bit_mask]; - FB_WRITEL((end_mask & eorx)^bgx, dst++); - if (!shift) { shift = 8; src++; } + for (i = image->height; i--; ) { + dst = (u32 __iomem *)dst1; + src = s; + + switch (ppw) { + case 4: /* 8 bpp */ + for (j = k; j; j -= 2, ++src) { + FB_WRITEL(colortab[(*src >> 4) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 0) & bit_mask], dst++); + } + break; + case 2: /* 16 bpp */ + for (j = k; j; j -= 4, ++src) { + FB_WRITEL(colortab[(*src >> 6) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 4) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 2) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 0) & bit_mask], dst++); + } + break; + case 1: /* 32 bpp */ + for (j = k; j; j -= 8, ++src) { + FB_WRITEL(colortab[(*src >> 7) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 6) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 5) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 4) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 3) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 2) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 1) & bit_mask], dst++); + FB_WRITEL(colortab[(*src >> 0) & bit_mask], dst++); + } + break; } + dst1 += p->fix.line_length; s += spitch; } From patchwork Wed Feb 23 19:38:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Zimmermann X-Patchwork-Id: 12757459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46B2FC433FE for ; Wed, 23 Feb 2022 19:38:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233923AbiBWTij (ORCPT ); Wed, 23 Feb 2022 14:38:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238574AbiBWTii (ORCPT ); Wed, 23 Feb 2022 14:38:38 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E989748E67 for ; Wed, 23 Feb 2022 11:38:09 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 82667218A8; Wed, 23 Feb 2022 19:38:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q745Q564KmJVNw2+KNdAukT4aGfTFfnZ3w3mvLB/yBI=; b=isQfLLZ9DpWAk2rSDGCL21KLsDOMYjD5QXdnu4s6oEK78zUxWmA9ty+4BUAudzzHYFTNdh d1u+jusnKFVDzayeXBGJPN9uCYyQrzHGHK6rjD96mW2oUSVSub5DxJZqn/JlHAHNbBcI0D BJBu+WrgvqZ3bNTwUQ29IeLxrCuiPiU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1645645088; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q745Q564KmJVNw2+KNdAukT4aGfTFfnZ3w3mvLB/yBI=; b=QKf6D2JNo1iAlFUvobGBifCYUIul/yOOjv9AMCU+uRB90EJ/zhGCRDSCyiIj/h75ZQEMQW VUvPjVnDxsnSJUDg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4978A13DAC; Wed, 23 Feb 2022 19:38:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0PHwECCNFmKfUgAAMHmgww (envelope-from ); Wed, 23 Feb 2022 19:38:08 +0000 From: Thomas Zimmermann To: daniel@ffwll.ch, deller@gmx.de, javierm@redhat.com, geert@linux-m68k.org, sam@ravnborg.org, kraxel@redhat.com, ppaalanen@gmail.com Cc: dri-devel@lists.freedesktop.org, linux-fbdev@vger.kernel.org, Thomas Zimmermann Subject: [PATCH v3 5/5] drm: Add TODO item for optimizing format helpers Date: Wed, 23 Feb 2022 20:38:04 +0100 Message-Id: <20220223193804.18636-6-tzimmermann@suse.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220223193804.18636-1-tzimmermann@suse.de> References: <20220223193804.18636-1-tzimmermann@suse.de> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org Add a TODO item for optimizing blitting and format-conversion helpers in DRM and fbdev. There's always demand for faster graphics output. Signed-off-by: Thomas Zimmermann Acked-by: Sam Ravnborg Reviewed-by: Javier Martinez Canillas --- Documentation/gpu/todo.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/Documentation/gpu/todo.rst b/Documentation/gpu/todo.rst index 7bf7f2111696..7f113c6a02dd 100644 --- a/Documentation/gpu/todo.rst +++ b/Documentation/gpu/todo.rst @@ -241,6 +241,28 @@ Contact: Thomas Zimmermann , Daniel Vetter Level: Advanced +Benchmark and optimize blitting and format-conversion function +-------------------------------------------------------------- + +Drawing to dispay memory quickly is crucial for many applications' +performance. + +On at least x86-64, sys_imageblit() is significantly slower than +cfb_imageblit(), even though both use the same blitting algorithm and +the latter is written for I/O memory. It turns out that cfb_imageblit() +uses movl instructions, while sys_imageblit apparently does not. This +seems to be a problem with gcc's optimizer. DRM's format-conversion +heleprs might be subject to similar issues. + +Benchmark and optimize fbdev's sys_() helpers and DRM's format-conversion +helpers. In cases that can be further optimized, maybe implement a different +algorithm, For micro-optimizations, use movl/movq instructions explicitly. +That might possibly require architecture specific helpers (e.g., storel() +storeq()). + +Contact: Thomas Zimmermann + +Level: Intermediate drm_framebuffer_funcs and drm_mode_config_funcs.fb_create cleanup -----------------------------------------------------------------