Fix and improve FXAA by illwieckz · Pull Request #1924 · DaemonEngine/Daemon

illwieckz · 2026-03-07T12:23:10Z

This is so good to have it I'm now considering it for 0.56.0.

Fix FXAA sampling (use GL_LINEAR instead of GL_NEAREST)
make r_FXAA a new-style cvar
move FXAA control knobs to fxaa_fp (we now have #insert)
run FXAA after the camera shader (after tone mapping and color conversion)
implement luminance-based FXAA (also add r_showLuminance)
add r_showFXAA

Fix #533 and fix #1701:

Original comment:

No urge. I was looking at it in hope I could find a quick fix, I haven't found a quick fix, this is as broken as before.

Anyway, the rework of this may be useful for when the fix for FXAA will be found.

make r_FXAA a new-style cvar
move FXAA control knobs to fxaa_fp (we now have #insert)
run FXAA after the camera shader (after tone mapping and color conversion)
implement luminance-based FXAA (also add r_showLuminance)
add r_showFXAA

The r_showFXAA option paints in red the pixels whose RGB color is modified by FxaaPixelShader(). Actually FxaaPixelShader() modifies random pixels (not at edges) and the result image is more buggy than without FXAA.

…color conversion)

illwieckz · 2026-03-07T13:07:08Z

I verified that we don't have strong differences with the original FXAA3_11.h, and differences are white space being modified and typos being fixed in our end.

I used that as reference:

https://github.com/hghdev/NVIDIAGameWorks-GraphicsSamples/blob/master/samples/es3-kepler/FXAA/FXAA3_11.h

Which I assume is supposedly a copy of (now broken link):

https://github.com/NVIDIAGameWorks/GraphicsSamples/blob/master/samples/es3-kepler/FXAA/FXAA3_11.h

I have no idea where our bug is living.

illwieckz · 2026-03-07T13:21:24Z

When doing this:

diff --git a/src/engine/renderer/glsl_source/fxaa3_11_fp.glsl b/src/engine/renderer/glsl_source/fxaa3_11_fp.glsl
index dd3757b08..c4ed80f73 100644
--- a/src/engine/renderer/glsl_source/fxaa3_11_fp.glsl
+++ b/src/engine/renderer/glsl_source/fxaa3_11_fp.glsl
@@ -1240,6 +1240,7 @@ FxaaFloat4 FxaaPixelShader(
 #if (FXAA_DISCARD == 1)
     return FxaaTexTop(tex, posM);
 #else
+    return vec4(1.0, 0.0, 0.0, 1.0);
     return FxaaFloat4(FxaaTexTop(tex, posM).xyz, lumaM);
 #endif
 }

We see that the edges are properly detected:

But the data written to those areas is garbage.

illwieckz · 2026-03-07T14:15:10Z

Since the edge detection worked, I assumed that was the sampling that was broken.

I tested using filterType_t::FT_LINEAR as imageParams.filterType in R_CreateCurrentRenderImage(), and it fixed it!

Without FXAA:

With FXAA:

Here are pixels modified by FXAA with FT_NEAREST (what is currently used in code):

Here are pixels modified by FXAA with FT_LINEAR:

This last screenshot is NOT the painting of pixels expected to be modified by FXAA by returning red as FXAA return value (like in previous comment), it is the painting of pixels actually modified by FXAA, by comparing the input and the output of the FXAA computation and painting in red what changed.

illwieckz · 2026-03-07T14:36:47Z

So I checked the history, and that FT_NEAREST was already used in our first commit: eb73a11

illwieckz · 2026-03-07T14:45:24Z

I'm now git blaming the XreaL code (the git reference below is from a locally converted repository), and our code:

	imageParams.filterType = filterType_t::FT_NEAREST;
	imageParams.wrapType = wrapTypeEnum_t::WT_CLAMP;

	tr.currentRenderImage[0] = R_CreateImage( "*currentRender0", nullptr, width, height, 1, imageParams );
	tr.currentRenderImage[1] = R_CreateImage( "*currentRender1", nullptr, width, height, 1, imageParams );

is ultimately deriving from:

commit af8e5637ae850c20f375ba198acbe2dc4060e4f1
Author: Robert “Tr3B” Beckebans <xxx>
Date:   Sun Sep 10 20:53:54 2006 +0000

    simplified render to texture

diff --git a/xreal/code/renderer/tr_image.c b/xreal/code/renderer/tr_image.c
index cafe1ba4..ec253ec2 100644
--- a/xreal/code/renderer/tr_image.c
+++ b/xreal/code/renderer/tr_image.c
@@ -4540,38 +4540,7 @@ static void R_CreateCurrentRenderImage(void)
 
 	data = ri.Hunk_AllocateTempMemory(width * height * 4);
 
-	tr.currentRenderImage = R_CreateImage("_currentRender", data, width, height, IF_NOPICMIP, FT_DEFAULT, WT_REPEAT);
-
-	ri.Hunk_FreeTempMemory(data);
-}
-
-static void R_CreateCurrentRenderLinearImage(void)
-{
-	int             width, height;
-	byte           *data;
-	
-	width = NearestPowerOfTwo(glConfig.vidWidth);
-	height = NearestPowerOfTwo(glConfig.vidHeight);
-
-	data = ri.Hunk_AllocateTempMemory(width * height * 4);
-
-	tr.currentRenderLinearImage = R_CreateImage("_currentRenderLinear", data, width, height, IF_NOPICMIP, FT_LINEAR, WT_REPEAT);
-
-	ri.Hunk_FreeTempMemory(data);
-}
-
-static void R_CreateCurrentRenderNearestImage(void)
-{
-	int             width, height;
-	byte           *data;
-	
-	width = NearestPowerOfTwo(glConfig.vidWidth);
-	height = NearestPowerOfTwo(glConfig.vidHeight);
-
-	data = ri.Hunk_AllocateTempMemory(width * height * 4);
-
-	tr.currentRenderNearestImage =
-		R_CreateImage("_currentRenderNearest", data, width, height, IF_NOPICMIP, FT_NEAREST, WT_REPEAT);
+	tr.currentRenderImage = R_CreateImage("_currentRender", data, width, height, IF_NOPICMIP, FT_NEAREST, WT_REPEAT);
 
 	ri.Hunk_FreeTempMemory(data);
 }
@@ -4673,8 +4642,6 @@ void R_CreateBuiltinImages(void)
 	R_CreateAttenuationXYImage();
 	R_CreateContrastRenderImage();
 	R_CreateCurrentRenderImage();
-	R_CreateCurrentRenderLinearImage();
-	R_CreateCurrentRenderNearestImage();
 	R_CreateCurrentRenderFBOImage();
 	R_CreatePortalRenderFBOImage();
 }
diff --git a/xreal/code/renderer/tr_local.h b/xreal/code/renderer/tr_local.h
index 14dcb2fa..5813e7cd 100644
--- a/xreal/code/renderer/tr_local.h
+++ b/xreal/code/renderer/tr_local.h
@@ -1726,10 +1726,9 @@ typedef struct
 	image_t        *identityLightImage;	// full of tr.identityLightByte
 	image_t        *noFalloffImage;
 	image_t        *attenuationXYImage;
+	
 	image_t        *contrastRenderImage;
 	image_t        *currentRenderImage;
-	image_t        *currentRenderLinearImage;
-	image_t        *currentRenderNearestImage;
 	
 	image_t        *currentRenderFBOImage[4];
 	image_t        *portalRenderFBOImage[4];
diff --git a/xreal/code/renderer/tr_shade.c b/xreal/code/renderer/tr_shade.c
index 34ef1b5c..7082fd48 100644
--- a/xreal/code/renderer/tr_shade.c
+++ b/xreal/code/renderer/tr_shade.c
@@ -2637,9 +2637,8 @@ static void Render_heatHaze(int stage)
 		
 		// capture current color buffer for u_CurrentMap
 		GL_SelectTexture(0);
-		GL_Bind(tr.currentRenderNearestImage);
-		qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderNearestImage->uploadWidth,
-							 tr.currentRenderNearestImage->uploadHeight);
+		GL_Bind(tr.currentRenderImage);
+		qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderImage->uploadWidth, tr.currentRenderImage->uploadHeight);
 							 
 		// clear color buffer
 		qglClear(GL_COLOR_BUFFER_BIT);
@@ -2668,7 +2667,7 @@ static void Render_heatHaze(int stage)
 		
 		// bind u_CurrentMap
 		GL_SelectTexture(1);
-		GL_Bind(tr.currentRenderNearestImage);
+		GL_Bind(tr.currentRenderImage);
 
 		DrawElements();
 		
@@ -2695,7 +2694,7 @@ static void Render_heatHaze(int stage)
 
 		// bind u_CurrentMap
 		GL_SelectTexture(0);
-		GL_Bind(tr.currentRenderNearestImage);
+		GL_Bind(tr.currentRenderImage);
 
 		// set 2D virtual screen size
 		qglPushMatrix();
@@ -2765,8 +2764,8 @@ static void Render_heatHaze(int stage)
 	}
 	else
 	{
-		GL_Bind(tr.currentRenderLinearImage);
-		qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderLinearImage->uploadWidth, tr.currentRenderLinearImage->uploadHeight);
+		GL_Bind(tr.currentRenderImage);
+		qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderImage->uploadWidth, tr.currentRenderImage->uploadHeight);
 	}
 	
 	// bind u_ContrastMap
@@ -2822,9 +2821,8 @@ static void Render_bloom(int stage)
 	qglUniform2fARB(tr.contrastShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
 	GL_SelectTexture(0);
-	GL_Bind(tr.currentRenderNearestImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderNearestImage->uploadWidth,
-						 tr.currentRenderNearestImage->uploadHeight);
+	GL_Bind(tr.currentRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderImage->uploadWidth, tr.currentRenderImage->uploadHeight);
 	qglMatrixMode(GL_TEXTURE);
 	qglLoadMatrixf(tess.svars.texMatrices[TB_COLORMAP]);
 	qglMatrixMode(GL_MODELVIEW);
@@ -2841,9 +2839,8 @@ static void Render_bloom(int stage)
 	qglUniform2fARB(tr.bloomShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
 	GL_SelectTexture(1);
-	GL_Bind(tr.currentRenderLinearImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderLinearImage->uploadWidth,
-						 tr.currentRenderLinearImage->uploadHeight);
+	GL_Bind(tr.contrastRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.contrastRenderImage->uploadWidth, tr.contrastRenderImage->uploadHeight);
 
 	DrawElements();
 
@@ -2877,9 +2874,8 @@ static void Render_bloom2(int stage)
 	qglUniform2fARB(tr.contrastShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
 	GL_SelectTexture(0);
-	GL_Bind(tr.currentRenderNearestImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderNearestImage->uploadWidth,
-						 tr.currentRenderNearestImage->uploadHeight);
+	GL_Bind(tr.currentRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderImage->uploadWidth, tr.currentRenderImage->uploadHeight);
 	qglMatrixMode(GL_TEXTURE);
 	qglLoadMatrixf(tess.svars.texMatrices[TB_COLORMAP]);
 	qglMatrixMode(GL_MODELVIEW);
@@ -2894,9 +2890,8 @@ static void Render_bloom2(int stage)
 	qglUniform2fARB(tr.blurXShader.u_FBufScale, fbufWidthScale, fbufHeightScale);
 	qglUniform2fARB(tr.blurXShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
-	GL_Bind(tr.currentRenderLinearImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderLinearImage->uploadWidth,
-						 tr.currentRenderLinearImage->uploadHeight);
+	GL_Bind(tr.contrastRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.contrastRenderImage->uploadWidth, tr.contrastRenderImage->uploadHeight);
 
 	DrawElements();
 
@@ -2908,9 +2903,8 @@ static void Render_bloom2(int stage)
 	qglUniform2fARB(tr.blurYShader.u_FBufScale, fbufWidthScale, fbufHeightScale);
 	qglUniform2fARB(tr.blurYShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
-	GL_Bind(tr.currentRenderLinearImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderLinearImage->uploadWidth,
-						 tr.currentRenderLinearImage->uploadHeight);
+	GL_Bind(tr.contrastRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.contrastRenderImage->uploadWidth, tr.contrastRenderImage->uploadHeight);
 
 	DrawElements();
 
@@ -2924,10 +2918,10 @@ static void Render_bloom2(int stage)
 	qglUniform2fARB(tr.bloomShader.u_NPOTScale, npotWidthScale, npotHeightScale);
 
 	GL_SelectTexture(0);
-	GL_Bind(tr.currentRenderNearestImage);
+	GL_Bind(tr.currentRenderImage);
 
 	GL_SelectTexture(1);
-	GL_Bind(tr.currentRenderLinearImage);
+	GL_Bind(tr.contrastRenderImage);
 
 	DrawElements();
 
@@ -2963,9 +2957,8 @@ static void Render_rotoscope(int stage)
 
 	// bind u_ColorMap
 	GL_SelectTexture(0);
-	GL_Bind(tr.currentRenderNearestImage);
-	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderNearestImage->uploadWidth,
-						 tr.currentRenderNearestImage->uploadHeight);
+	GL_Bind(tr.currentRenderImage);
+	qglCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0, tr.currentRenderImage->uploadWidth, tr.currentRenderImage->uploadHeight);
 	qglMatrixMode(GL_TEXTURE);
 	qglLoadMatrixf(tess.svars.texMatrices[TB_COLORMAP]);
 	qglMatrixMode(GL_MODELVIEW);

Before that XreaL commit, the code had some tr.currentRenderNearestImage and tr.currentRenderLinearImage, I don't know yet how this was used. After that commit, the currentRender is always using FT_NEAREST.

illwieckz · 2026-03-07T14:48:50Z

This usage of FT_NEAREST for currentRender predates our implementation of FXAA, so it never worked.

- fix FXAA by using GL_LINEAR on currentRender, - restore GL_NEAREST after that to not break other effects.

illwieckz · 2026-03-07T20:08:25Z

Well, it works until we use bindless textures, and it's not suprising since the tweak requires a bind…

illwieckz · 2026-03-07T20:12:30Z

@VReaperV I would appreciate your expertise on this ! 🙂️

The way to fix FXAA is to make sure the framebuffer is using GL_LINEAR as filter, but the said framebuffer is set to use GL_NEAREST by default. I assume other effects require GL_NEAREST (bloom or motion blur, maybe), so we need to switch the filter mode as far as I understand how it works.

I verified that in non-bindless mode, switching to GL_LINEAR before running the FXAA is doing the job. Now I don't really know how to test if the switching back to GL_NEAREST is working.

And something I don't know how to do and what to do at all, is how this interacts with the bindless pipeline and how to make it work the bindless way.

slipher · 2026-03-07T23:47:03Z

If you're OK with requiring OpenGL 3.3 you can use "sampler" objects.

Starting from OpenGL 3.3 Core Profile, you can use Sampler Objects to specify the sampling parameters. When a sample object is bound to a texture image unit, it basically overrides all internal sampler settings of the bound texture object.

https://stackoverflow.com/a/73445598

illwieckz · 2026-03-07T23:54:24Z

Nice!

illwieckz · 2026-03-08T00:11:46Z

I tried using glBindSampler() and it works the non-bindless way, but it still doesn't work the bindless way.

illwieckz · 2026-03-08T00:51:09Z

It looks like I got it working with bindless textures, but I don't know what I am doing. 🫣️

illwieckz · 2026-03-08T01:01:59Z

If it works properly, maybe we will enable FXAA in high preset, and then by default.

We may want to disable FXAA internally when MSAA is enabled.

slipher · 2026-03-08T07:31:45Z

src/engine/renderer/glsl_source/cameraEffects_fp.glsl

+		// That luminance vector comes from a comment in fxaa3_11_fp.glsl.
+		vec3 luminanceVector = vec3( 0.299, 0.587, 0.114 );
+
+		float luminance = dot( color.rgb, luminanceVector );


Apparently it should be called "luma" since you're doing it on sRGB values. While "luminance" would be with linear values

slipher · 2026-03-08T08:11:16Z

Well, it works as intended against jaggies. Nice job! On the other hand, it makes a hash of any kind of texture that has a lot of lines or other fine detail. Also it likes to erase specular highlights. So it's good to have as an option, but I'm not convinced that things look better overall. Nothing against the PR itself, but I lacked a convenient place to discuss the question of whether it should be a part of the default presets.

Fixing jaggies in the door:

Blurring the plat23 floor texture:

Blurring the grate with the green liquid below; also the result seems too bright:

Blurring everywhere on the drill model:

illwieckz added 5 commits March 7, 2026 00:20

renderer: make r_FXAA a new-style cvar

d8ecd33

glsl: move FXAA control knobs to fxaa_fp

bf076ba

tr_backend: run FXAA after the camera shader (after tone mapping and …

e6c9f58

…color conversion)

renderer: implement luminance-based FXAA

60fc015

renderer: add r_showFXAA

a02cb5e

illwieckz marked this pull request as draft March 7, 2026 12:23

illwieckz force-pushed the illwieckz/fxaa branch 3 times, most recently from f5e4556 to d7013c8 Compare March 7, 2026 13:03

illwieckz force-pushed the illwieckz/fxaa branch from d7013c8 to a02cb5e Compare March 7, 2026 13:09

tr_backend: fix FXAA sampling by using GL_LINEAR on currentRender

2931004

- fix FXAA by using GL_LINEAR on currentRender, - restore GL_NEAREST after that to not break other effects.

illwieckz changed the title ~~WIP: improvements over FXAA~~ Fix and improve FXAA Mar 7, 2026

illwieckz marked this pull request as ready for review March 7, 2026 15:11

illwieckz force-pushed the illwieckz/fxaa branch from d505fbf to 2931004 Compare March 7, 2026 15:13

illwieckz mentioned this pull request Mar 7, 2026

Nuke FXAA? #1909

Closed

illwieckz requested a review from VReaperV March 7, 2026 20:12

fixup: use glBindSampler()

24a5f26

illwieckz force-pushed the illwieckz/fxaa branch 2 times, most recently from 2be5a7e to 24a5f26 Compare March 8, 2026 00:14

fixup: make it working with bindless textures

24fe6b1

illwieckz force-pushed the illwieckz/fxaa branch from 3ca219b to 24fe6b1 Compare March 8, 2026 00:58

slipher reviewed Mar 8, 2026

View reviewed changes

Conversation

illwieckz commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

illwieckz commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

illwieckz commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

illwieckz commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

illwieckz commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slipher commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 7, 2026

Uh oh!

illwieckz commented Mar 8, 2026

Uh oh!

illwieckz commented Mar 8, 2026

Uh oh!

illwieckz commented Mar 8, 2026

Uh oh!

slipher Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

slipher commented Mar 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

illwieckz commented Mar 7, 2026 •

edited

Loading

illwieckz commented Mar 7, 2026 •

edited

Loading

illwieckz commented Mar 7, 2026 •

edited

Loading

illwieckz commented Mar 7, 2026 •

edited

Loading

illwieckz commented Mar 7, 2026 •

edited

Loading