Replace JSON.NET with System.Text.Json across the codebase by niemyjski · Pull Request #2135 · exceptionless/Exceptionless

niemyjski · 2026-02-28T22:53:05Z

Replace JSON.NET with System.Text.Json

Replaces all Newtonsoft.Json serialization with System.Text.Json (STJ) and migrates from NEST to Elastic.Clients.Elasticsearch.

Key changes

Serialization

All JSON serialization now uses STJ with SnakeCaseLower naming policy
ObjectToInferredTypesConverter handles object-typed properties (replaces DataObjectConverter)
[JsonExtensionData] on Event merges root-level @error, @request, etc. into Data dict
[JsonPropertyName] overrides for legacy property names (o_s_name, o_s_version)
V1 webhook models use [JsonPropertyName] for PascalCase backward compatibility

Elasticsearch

Migrated from NEST to Elastic.Clients.Elasticsearch throughout
All index configurations use new fluent API
Migrations and jobs updated for new ES client
Repository queries migrated to Foundatio expression-based API

Event processing

All event upgraders migrated from JObject to JsonObject (System.Text.Json.Nodes)
All plugins migrated from Newtonsoft serializer to STJ
GetValue<T>() simplified — removed unnecessary TryDeserializeWithFallback (PascalCase data never existed in ES)
Error write-back fix in ErrorPlugin / SimpleErrorPlugin after pipeline mutation

Removed

DataObjectConverter, ElasticJsonNetSerializer, all Newtonsoft classes
Foundatio.JsonNet, NEST.JsonNetSerializer, FluentRest.NewtonsoftJson packages
Unnecessary PascalCase fallback machinery

Test results

1549 total, 0 failed, 1547 passed, 2 skipped
New: 27 serializer tests, model serialization tests, query migration tests, integration tests

Copilot

Pull request overview

Migrates application-layer JSON usage from Newtonsoft.Json to System.Text.Json (STJ), aligning core models/plugins/tests and adding STJ-based infrastructure for Elasticsearch/NEST while keeping Newtonsoft only as a transitive dependency for the NEST wire protocol.

Changes:

Introduces STJ-based IElasticsearchSerializer implementation and STJ type metadata modifiers to match prior Newtonsoft serialization behavior (e.g., omit empty collections).
Updates core pipeline/plugins/controllers/jobs to use ITextSerializer/JsonSerializerOptions instead of Newtonsoft abstractions.
Refactors tests and fixtures to validate STJ semantics (including semantic JSON comparison).

Reviewed changes

Copilot reviewed 87 out of 87 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/Exceptionless.Tests/Utility/DataBuilder.cs	Updates test builder to use `ITextSerializer` for manual stacking info.
tests/Exceptionless.Tests/Serializer/SerializerTests.cs	Reworks serializer tests away from Newtonsoft-specific behavior toward STJ round-trips.
tests/Exceptionless.Tests/Serializer/Models/PersistentEventSerializerTests.cs	Uses `ITextSerializer` for typed data retrieval in PersistentEvent tests.
tests/Exceptionless.Tests/Serializer/Models/DataDictionaryTests.cs	Updates `GetValue<T>` tests to use `ITextSerializer` pathway.
tests/Exceptionless.Tests/Repositories/StackRepositoryTests.cs	Compares serialized JSON via `ITextSerializer` instead of `ToJson()`.
tests/Exceptionless.Tests/Repositories/ProjectRepositoryTests.cs	Updates Slack token access to use serializer-aware accessor.
tests/Exceptionless.Tests/Repositories/EventRepositoryTests.cs	Uses serializer-based typed accessors in repository tests.
tests/Exceptionless.Tests/Plugins/WebHookDataTests.cs	Switches expected/actual comparisons to semantic JSON equivalence using STJ.
tests/Exceptionless.Tests/Plugins/SummaryDataTests.cs	Moves summary comparisons to semantic JSON using STJ.
tests/Exceptionless.Tests/Plugins/GeoTests.cs	Updates Geo plugin/tests to pass `ITextSerializer`.
tests/Exceptionless.Tests/Plugins/EventUpgraderTests.cs	Uses `JsonNode` formatting + semantic compare for upgrader outputs.
tests/Exceptionless.Tests/Plugins/EventParserTests.cs	Uses `ITextSerializer` for round-trip verification; removes Newtonsoft formatting asserts.
tests/Exceptionless.Tests/Pipeline/EventPipelineTests.cs	Updates pipeline tests to use serializer-based typed accessors.
tests/Exceptionless.Tests/Mail/MailerTests.cs	Updates Mailer construction to accept `ITextSerializer`.
tests/Exceptionless.Tests/IntegrationTestsBase.cs	Replaces FluentRest Newtonsoft serializer with STJ `JsonContentSerializer`.
tests/Exceptionless.Tests/Exceptionless.Tests.csproj	Removes `FluentRest.NewtonsoftJson`, adds STJ-based `FluentRest`.
tests/Exceptionless.Tests/Controllers/EventControllerTests.cs	Uses `ITextSerializer` for typed accessors in controller tests.
src/Exceptionless.Web/Exceptionless.Web.csproj	Removes `NEST.JsonNetSerializer` package reference.
src/Exceptionless.Web/Controllers/ProjectController.cs	Injects `ITextSerializer` for Slack token access.
src/Exceptionless.Web/Controllers/EventController.cs	Injects `ITextSerializer` and uses it for event payload byte generation.
src/Exceptionless.Core/Utility/TypeHelper.cs	Updates equality handling for STJ `JsonElement`.
src/Exceptionless.Core/Utility/ExtensibleObject.cs	Adds `JsonElement` handling to generic property retrieval.
src/Exceptionless.Core/Utility/ErrorSignature.cs	Switches error signature extraction to serializer-based `GetValue<T>`.
src/Exceptionless.Core/Services/SlackService.cs	Uses serializer-aware Slack token access.
src/Exceptionless.Core/Serialization/ObjectToInferredTypesConverter.cs	Refines number inference logic for STJ dynamic object conversion.
src/Exceptionless.Core/Serialization/LowerCaseUnderscorePropertyNamesContractResolver.cs	Deletes Newtonsoft contract resolver (obsolete after STJ migration).
src/Exceptionless.Core/Serialization/JsonSerializerOptionsExtensions.cs	Defines Exceptionless STJ defaults (snake_case, safe encoder, converters, empty-collection skipping).
src/Exceptionless.Core/Serialization/ExceptionlessNamingStrategy.cs	Deletes Newtonsoft naming strategy (replaced by STJ naming policy).
src/Exceptionless.Core/Serialization/EmptyCollectionModifier.cs	Adds STJ type info modifier to omit empty collections.
src/Exceptionless.Core/Serialization/ElasticSystemTextJsonSerializer.cs	Adds STJ `IElasticsearchSerializer` for NEST with custom converters.
src/Exceptionless.Core/Serialization/ElasticJsonNetSerializer.cs	Deletes Newtonsoft-based NEST serializer.
src/Exceptionless.Core/Serialization/ElasticConnectionSettingsAwareContractResolver.cs	Deletes Newtonsoft resolver used by old NEST serializer.
src/Exceptionless.Core/Serialization/DynamicTypeContractResolver.cs	Deletes Newtonsoft dynamic contract resolver.
src/Exceptionless.Core/Serialization/DataObjectConverter.cs	Deletes Newtonsoft-based model/data converter.
src/Exceptionless.Core/Repositories/Configuration/ExceptionlessElasticConfiguration.cs	Switches Elasticsearch client wiring to STJ serializer.
src/Exceptionless.Core/Plugins/WebHook/Default/010_VersionOnePlugin.cs	Uses serializer-based typed accessors; adds `[JsonPropertyName]` for v1 webhook compatibility.
src/Exceptionless.Core/Plugins/WebHook/Default/005_SlackPlugin.cs	Uses serializer-based typed accessors for Slack webhook creation.
src/Exceptionless.Core/Plugins/Formatting/FormattingPluginBase.cs	Converts formatting plugins to depend on `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/99_DefaultFormattingPlugin.cs	Updates default formatting/slack attachment creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/60_LogFormattingPlugin.cs	Updates typed accessors + slack attachment creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/50_SessionFormattingPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/40_UsageFormattingPlugin.cs	Updates typed accessors + slack attachment creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/30_NotFoundFormattingPlugin.cs	Updates typed accessors + IP retrieval to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/20_ErrorFormattingPlugin.cs	Updates typed accessors + slack attachment creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/10_SimpleErrorFormattingPlugin.cs	Updates typed accessors + slack attachment creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/Formatting/Default/05_ManualStackingFormattingPlugin.cs	Updates manual stacking info access to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventUpgrader/EventUpgraderContext.cs	Migrates upgrader DOM from `JObject/JArray` to `JsonObject/JsonArray`.
src/Exceptionless.Core/Plugins/EventUpgrader/Default/V2_EventUpgrade.cs	Rewrites upgrader logic from Newtonsoft DOM to `JsonNode` DOM.
src/Exceptionless.Core/Plugins/EventUpgrader/Default/V1R850_EventUpgrade.cs	Rewrites upgrader logic to `JsonObject`.
src/Exceptionless.Core/Plugins/EventUpgrader/Default/V1R844_EventUpgrade.cs	Rewrites upgrader logic to `JsonObject`.
src/Exceptionless.Core/Plugins/EventUpgrader/Default/V1R500_EventUpgrade.cs	Rewrites upgrader logic to `JsonObject` and normalizes date string formatting.
src/Exceptionless.Core/Plugins/EventUpgrader/Default/GetVersion.cs	Rewrites version detection to `JsonNode` APIs.
src/Exceptionless.Core/Plugins/EventProcessor/Default/90_RemovePrivateInformationPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/80_AngularPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/70_SessionPlugin.cs	Updates typed accessors + session start creation to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/50_GeoPlugin.cs	Updates IP extraction to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/45_EnvironmentInfoPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/40_RequestInfoPlugin.cs	Updates typed accessors and request exclusion processing to accept `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/30_SimpleErrorPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/20_ErrorPlugin.cs	Updates typed accessors + ErrorSignature to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/10_NotFoundPlugin.cs	Updates typed accessors to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/0_ThrottleBotsPlugin.cs	Updates request-info access to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventProcessor/Default/03_ManualStackingPlugin.cs	Updates manual stacking info access to use `ITextSerializer`.
src/Exceptionless.Core/Plugins/EventParser/Default/LegacyErrorParserPlugin.cs	Migrates legacy parsing to `JsonNode` + STJ options.
src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs	Uses `ITextSerializer` for parsing event payloads instead of Newtonsoft.
src/Exceptionless.Core/Models/SummaryData.cs	Adjusts nullability/required-ness to align with STJ behaviors.
src/Exceptionless.Core/Models/StackSummaryModel.cs	Adjusts nullability/required-ness to align with STJ behaviors.
src/Exceptionless.Core/Models/Stack.cs	Removes Newtonsoft enum converter attribute; keeps STJ converter.
src/Exceptionless.Core/Models/SlackToken.cs	Replaces Newtonsoft `[JsonProperty]` with STJ `[JsonPropertyName]`; updates SlackAttachment ctor to use `ITextSerializer`.
src/Exceptionless.Core/Models/Messaging/ReleaseNotification.cs	Removes `required` modifiers (likely to avoid STJ required-member enforcement).
src/Exceptionless.Core/Models/Event.cs	Adds `[JsonExtensionData]` + `IJsonOnDeserialized` merge into `Data`.
src/Exceptionless.Core/Mail/Mailer.cs	Switches user-info extraction to serializer-based accessors.
src/Exceptionless.Core/Jobs/WebHooksJob.cs	Switches webhook POST payload handling to STJ `PostAsJsonAsync` + options.
src/Exceptionless.Core/Jobs/EventPostsJob.cs	Uses serializer-based event bytes for retries.
src/Exceptionless.Core/Jobs/EventNotificationsJob.cs	Updates request-info access to use `ITextSerializer`.
src/Exceptionless.Core/Jobs/CloseInactiveSessionsJob.cs	Updates identity extraction to use `ITextSerializer`.
src/Exceptionless.Core/Extensions/RequestInfoExtensions.cs	Requires serializer for post-data exclusion parsing (STJ).
src/Exceptionless.Core/Extensions/ProjectExtensions.cs	Makes Slack token extraction serializer-aware via `GetValue<T>`.
src/Exceptionless.Core/Extensions/PersistentEventExtensions.cs	Updates helpers to accept `ITextSerializer` for typed accessors.
src/Exceptionless.Core/Extensions/JsonNodeExtensions.cs	Adds STJ JsonNode helpers used by upgraders/tests (formatting, mutation helpers).
src/Exceptionless.Core/Extensions/JsonExtensions.cs	Removes Newtonsoft helpers, retains JSON-type detection helpers.
src/Exceptionless.Core/Extensions/EventExtensions.cs	Updates typed accessor APIs to accept `ITextSerializer`; switches `GetBytes` to serializer bytes.
src/Exceptionless.Core/Extensions/ErrorExtensions.cs	Updates stacking target helper to use serializer-based error access.
src/Exceptionless.Core/Extensions/DataDictionaryExtensions.cs	Reworks `GetValue<T>` around STJ/`ITextSerializer`, adds case-insensitive JsonElement handling.
src/Exceptionless.Core/Exceptionless.Core.csproj	Removes Newtonsoft/JsonNet packages.
src/Exceptionless.Core/Bootstrapper.cs	Removes Newtonsoft setup; registers STJ options + `SystemTextJsonSerializer` in DI.
AGENTS.md	Documents the new STJ-only serialization architecture and security posture.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 87 out of 87 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Remove all Newtonsoft.Json serialization from the application layer, replacing it with System.Text.Json (STJ). NEST still brings in Newtonsoft transitively, but all application-level serialization now uses STJ exclusively. Key changes: - Add ElasticSystemTextJsonSerializer as custom IElasticsearchSerializer for NEST - Add EmptyCollectionModifier to omit empty collections during serialization - Add ObjectToInferredTypesConverter to handle JObject/JToken from NEST reads - Add JsonNodeExtensions as STJ equivalents of JObject helpers for event upgraders - Add IJsonOnDeserialized to Event model to merge [JsonExtensionData] into Data dict - Add [JsonPropertyName] attributes to V1 webhook models for PascalCase compat - Migrate all event upgraders from JObject to JsonObject (System.Text.Json.Nodes) - Migrate all plugins from ISerializer/JsonSerializerOptions DI injection - Use case-insensitive deserialization for DataDictionary.GetValue<T>() from JsonElement - Use semantic comparison (JsonNode.DeepEquals) in tests for fixture validation - Remove DataObjectConverter, ElasticJsonNetSerializer, and related Newtonsoft classes - Remove Foundatio.JsonNet, NEST.JsonNetSerializer, FluentRest.NewtonsoftJson packages Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Fix ConvertJsonElement ternary type coercion: (object)l cast prevents implicit long→double widening in switch expression - Make ObjectToInferredTypesConverter configurable with preferInt64 flag for ES serializer to match JSON.NET DataObjectConverter behavior - Fix ElasticSystemTextJsonSerializer: remove ReadStreamToSpan lifetime bug (span backed by disposed MemoryStream), deserialize from stream directly with MemoryStream fast-path - Fix Serialize<T> indentation: pass JsonWriterOptions to Utf8JsonWriter so SerializationFormatting.Indented actually produces indented output - Handle exponent notation (1e5) as floating-point in ReadNumber - Use double consistently (not decimal) for floating-point to match JSON.NET behavior - Fix RenameAll return value: return whether any renames occurred - Add using var to MemoryStream in EventController and EventPostsJob - Handle empty response bodies in SendRequestAsAsync (STJ throws on empty input, Newtonsoft returned default) - Fix SerializerTests: put unknown properties at root level to test JsonExtensionData→ConvertJsonElement path correctly - Revert AGENTS.md to main Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 86 out of 86 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…icsearch Replace IElasticsearchSerializer with Elastic.Transport.Serializer, update ExceptionlessElasticConfiguration to use ElasticsearchClient/ElasticsearchClientSettings/ StaticNodePool, pass ITextSerializer to Foundatio base, and register ElasticsearchClient explicitly in DI.

Update all index files to use CreateIndexRequestDescriptor, void returns for ConfigureIndex/ConfigureIndexMapping, expression-based property mappings, DynamicMapping enum, and renamed analysis methods.

Replace QueryContainer with Query, use TermQuery/TermsQuery/BoolQuery/DateRangeQuery object initializers with Infer.Field expressions.

…earch Update CleanupOrphanedDataJob, DataMigrationJob, all migration files, AdminController, EventController, ProjectController, and test files to use new ES 8 client APIs.

Copilot

Pull request overview

Copilot reviewed 117 out of 117 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

SetupDefaults() already maps the Id property. The explicit Keyword(e => e.Id) mapping caused a duplicate key error in the new Elastic.Clients.Elasticsearch client.

- Fix StackRepository to use lowercase enum string for FieldEquals - Quote ISO 8601 dates in Lucene filter expressions (EventRepository) - Fix EmptyCollectionModifier to only skip null, not empty collections - Add SnakeCaseOptions for DataDictionary deserialization compatibility - Update event serialization test fixture for UTC Z suffix - Use order-independent JSON comparison in serialization test 5 PersistentEventQueryValidatorTests still fail due to a Foundatio.Parsers bug with grouped TermRangeNode field inheritance. Requires new Foundatio.Parsers preview package.

niemyjski · 2026-05-13T02:08:25Z

+            query = query.ElasticFilter(new DateRangeQuery { Field = ElasticInfer.Field<PersistentEvent>(e => e.Date), Lt = utcEnd.Value.ToString("O") });
        else if (utcStart.HasValue)
-            query = query.ElasticFilter(Query<PersistentEvent>.DateRange(r => r.Field(e => e.Date).GreaterThan(utcStart)));
+            query = query.ElasticFilter(new DateRangeQuery { Field = ElasticInfer.Field<PersistentEvent>(e => e.Date), Gt = utcStart.Value.ToString("O") });


do we really have to convert to string?

niemyjski · 2026-05-13T02:08:41Z

-            .ElasticFilter(!Query<PersistentEvent>.Ids(ids => ids.Values(ev.Id)))
-            .FilterExpression(String.Concat(EventIndex.Alias.StackId, ":", ev.StackId))
+            .Stack(ev.StackId)
+            .ElasticFilter(new BoolQuery { MustNot = [new TermQuery { Field = "_id", Value = ev.Id }] })


can we infer this magic string

niemyjski · 2026-05-13T02:08:47Z

-            .ElasticFilter(!Query<PersistentEvent>.Ids(ids => ids.Values(ev.Id)))
-            .FilterExpression(String.Concat(EventIndex.Alias.StackId, ":", ev.StackId))
+            .Stack(ev.StackId)
+            .ElasticFilter(new BoolQuery { MustNot = [new TermQuery { Field = "_id", Value = ev.Id }] })


can we infer this magic string

niemyjski · 2026-05-13T02:10:45Z

+        var query = new RepositoryQuery<Organization>();
+
        if (!String.IsNullOrWhiteSpace(criteria))
-            filter &= (Query<Organization>.Term(o => o.Id, criteria) || Query<Organization>.Term(o => o.Name, criteria));


Can we use the new Foundatio.Repository Query Grouping for any for these repository queries like this since we now support grouping would be nice to use that instead of elastic filters? Not sure we have minimumShouldMatch? We need to make sure we have test coverage on all these repo queries

niemyjski · 2026-05-13T02:11:05Z

            //    break;
            default:
-                query.SortAscending(o => o.Name.Suffix("keyword"));
+                query.SortAscending((Field)"name.keyword");


kinda shocked the parsers wouldn't use the keyword field by default?

and we use an expression for this?>

niemyjski · 2026-05-13T02:11:29Z

            return Task.FromResult(new FindResults<Project>());

-        return FindAsync(q => q.Organization(organizationIds).SortAscending(p => p.Name.Suffix("keyword")), options);
+        return FindAsync(q => q.Organization(organizationIds).SortAscending((Field)"name.keyword"), options);


expression for this?

niemyjski · 2026-05-13T02:11:56Z

-        return FindAsync(q => q.ElasticFilter(filter).SortAscending(p => p.OrganizationId), o => o.SearchAfterPaging().PageLimit(limit));
+        long threshold = _timeProvider.GetUtcNow().UtcDateTime.Ticks - (TimeSpan.TicksPerHour * hourToSendNotificationsAfterUtcMidnight);
+        return FindAsync(q => q
+            .ElasticFilter(new NumberRangeQuery { Field = ElasticInfer.Field<Project>(p => p.NextSummaryEndOfDayTicks), Lt = threshold })


can we use the new foundatio query ranges

niemyjski · 2026-05-13T02:12:18Z

            .Organization(organizationId)
            .FieldEquals(e => e.ViewType, viewType)
-            .SortAscending(e => e.Name.Suffix("keyword")), options);
+            .SortAscending((Field)"name.keyword"), options);


look at all of this can we use the sort expressions, keyword shoudl be picked up, look at all usages of sorts.

niemyjski · 2026-05-13T02:12:39Z

+            // Pass the serialized enum string directly because Foundatio's FieldValueHelper.ToFieldValue
+            // calls .ToString() on enums (producing "Open") instead of respecting [JsonStringEnumMemberName("open")].
+            .FieldEquals(f => f.Status, "open")


thius is a hack, figure this out.

niemyjski · 2026-05-13T02:13:27Z


        emailAddress = emailAddress.Trim().ToLowerInvariant();
-        var hit = await FindOneAsync(q => q.ElasticFilter(Query<User>.Term(u => u.EmailAddress.Suffix("keyword"), emailAddress)), o => o.Cache(EmailCacheKey(emailAddress)));
+        var hit = await FindOneAsync(q => q.FieldEquals((Field)"email_address.keyword", emailAddress), o => o.Cache(EmailCacheKey(emailAddress)));


field expression? this should pick the keyword field by default.

niemyjski · 2026-05-13T02:14:16Z

+
+        // XSS-safe encoder: escapes <, >, &, ' while allowing Unicode characters
+        // This protects against script injection when JSON is embedded in HTML/JavaScript
+        options.Encoder = JavaScriptEncoder.Create(new TextEncoderSettings(UnicodeRanges.All));


where exactly do we need this for?

niemyjski · 2026-05-13T02:14:35Z

        options.RespectNullableAnnotations = true;

+        // Skip empty collections during serialization to match Newtonsoft behavior
+        options.TypeInfoResolver = new DefaultJsonTypeInfoResolver


where do we use this functinality, give examples

niemyjski · 2026-05-13T02:15:22Z

@@ -43,6 +43,25 @@ namespace Exceptionless.Core.Serialization;
 /// <seealso href="https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/converters-how-to#deserialize-inferred-types-to-object-properties"/>
 public sealed class ObjectToInferredTypesConverter : JsonConverter<object?>


do we need this or can we use the elastic repos one?

niemyjski · 2026-05-13T02:16:25Z

-    public static DynamicTypeContractResolver GetJsonContractResolver()
-    {
-        var resolver = new DynamicTypeContractResolver(new LowerCaseUnderscorePropertyNamesContractResolver());
-        resolver.UseDefaultResolverFor(typeof(DataDictionary), typeof(SettingsDictionary), typeof(VersionOnePlugin.VersionOneWebHookStack), typeof(VersionOnePlugin.VersionOneWebHookEvent));


make sure these types are still serialized correctly. do we have test coverage with models for this type on a complex object that passed before changes in this branch.

niemyjski · 2026-05-13T02:17:52Z

    await page.goto('/next');
    await page.waitForURL('/next/login');
-    await expect(page.getByRole('button', { name: 'Login' })).toBeVisible();
+    await expect(page.getByRole('button', { name: 'Login', exact: true })).toBeVisible();


fix this check failure Check failure on line R6
Check failure:
Expected "exact" to come before "name"

niemyjski · 2026-05-13T02:19:34Z

    "stack_id": "<STACK_ID>",
    "is_first_occurrence": true,
-    "created_utc": "2026-01-15T12:00:00",
+    "created_utc": "2026-01-15T12:00:00Z",


is this correct? is this ISO8601?

niemyjski · 2026-05-13T02:19:58Z

-        Assert.NotNull(snapshots.Repositories);
-        Assert.NotNull(snapshots.Snapshots);


should this have changed?

niemyjski · 2026-05-13T02:20:41Z

+                var obj = new JsonObject();
+                foreach (var prop in element.EnumerateObject().OrderBy(p => p.Name, StringComparer.Ordinal))
+                    obj[prop.Name] = SortElement(prop.Value);
+                return obj;


new line before return

niemyjski · 2026-05-13T02:20:46Z

+                var arr = new JsonArray();
+                foreach (var item in element.EnumerateArray())
+                    arr.Add(SortElement(item));
+                return arr;


new line before return

niemyjski · 2026-05-13T02:21:35Z


        var events = _parser.ParseEvents(json, 2, "exceptionless/2.0.0.0");
        Assert.Single(events);
+        var ev = events.First();


Suggested change

var ev = events.First();

var ev = events.Single();

niemyjski · 2026-05-13T02:23:31Z

+    [InlineData("data.age:(>30 AND <=40)", "idx.age-n:(idx.age-n:>30 AND idx.age-n:<=40)", true, true)]
+    [InlineData("data.age:(+>=10 AND < 20)", "idx.age-n:(+idx.age-n:>=10 AND idx.age-n:<20)", true, true)]
+    [InlineData("data.age:(+>=10 +<20)", "idx.age-n:(+idx.age-n:>=10 AND +idx.age-n:<20)", true, true)]
+    [InlineData("data.age:(->=10 AND < 20)", "idx.age-n:(-idx.age-n:>=10 AND idx.age-n:<20)", true, true)]
    [InlineData("data.age:[10 TO *]", "idx.age-n:[10 TO *]", true, true)]
    [InlineData("data.age:[* TO 10]", "idx.age-n:[* TO 10]", true, true)]
-    [InlineData("type:404 AND data.age:(>30 AND <=40)", "type:404 AND idx.age-n:(>30 AND <=40)", true, true)]
+    [InlineData("type:404 AND data.age:(>30 AND <=40)", "type:404 AND idx.age-n:(idx.age-n:>30 AND idx.age-n:<=40)", true, true)]


why did this have to changed, it was already scoped. this feels wrong

niemyjski · 2026-05-13T02:24:46Z

+        var dict = new Dictionary<string, object?>
        {
-            HttpMethod = "GET",
-            Path = "/api/test",
-            Host = "localhost",
-            Port = 443,
-            IsSecure = true,
-            ClientIpAddress = "127.0.0.1"
-        });
-        var data = new DataDictionary { { "@request", jObject } };
+            ["http_method"] = "GET",
+            ["path"] = "/api/test",
+            ["host"] = "localhost",
+            ["port"] = 443,
+            ["is_secure"] = true,
+            ["client_ip_address"] = "127.0.0.1"
+        };


I don't like that this changed, I feel like this is masking some kind of issue.

niemyjski · 2026-05-13T02:24:57Z

+            ["machine_name"] = "TEST-MACHINE",
+            ["processor_count"] = 8,
+            ["total_physical_memory"] = 16000000000L,
+            ["o_s_name"] = "Windows",
+            ["o_s_version"] = "10.0"
+        };


same with this, waht is this masking,

niemyjski · 2026-05-13T02:25:31Z

+    }
+
+    [Fact]
+    public void DataDictionary_GetValue_InnerError_FromDictionary()


four part name, this should have three parts.

niemyjski · 2026-05-13T02:26:01Z

+    }
+
+    [Fact]
+    public void DataDictionary_GetValue_Method_FromDictionary()


four part name, this should have three parts.

niemyjski · 2026-05-13T02:26:57Z

    }

+    [Fact]
+    public void CanDeserializeEventWithData()


three part name

niemyjski · 2026-05-13T02:27:21Z


    [Fact]
-    public void CanDeserializeEventWithInvalidKnownDataTypes()
+    public void CanRoundTripEventWithKnownDataTypes()


three part name

niemyjski · 2026-05-13T02:27:51Z

+    }
+
+    [Fact]
+    public void Delta_Deserialize_PartialUpdate_OnlyTracksProvidedProperties()


four part name, this should have three parts.

+        var settings = new ElasticsearchClientSettings(
+            connectionPool,
+            sourceSerializer: (_, clientSettings) =>
+                new DefaultSourceSerializer(clientSettings, options =>
+                {
+                    // Base defaults from DI + Foundatio
+                    options.ConfigureExceptionlessDefaults();
+                    options.ConfigureFoundatioRepositoryDefaults();
+
+                    // ES-specific overrides (legacy data compatibility)
+                    options.RespectNullableAnnotations = false;
+
+                    // Remove JsonStringEnumConverter added by Foundatio: most enums store as integers in ES.
+                    // Exception: StackStatus has a type-level [JsonConverter(typeof(JsonStringEnumConverter))]
+                    // which takes precedence and stores as string (mapped as Keyword in StackIndex).
+                    for (int i = options.Converters.Count - 1; i >= 0; i--)
+                    {
+                        if (options.Converters[i] is System.Text.Json.Serialization.JsonStringEnumConverter)
+                            options.Converters.RemoveAt(i);
+                    }
+
+                    // ES needs all integers as long to match the old JSON.NET DataObjectConverter behavior.
+                    // Remove existing ObjectToInferredTypesConverter instances (from both Configure calls)
+                    // and insert preferInt64: true version at position 0 so STJ picks it first.
+                    for (int i = options.Converters.Count - 1; i >= 0; i--)
+                    {
+                        if (options.Converters[i] is Exceptionless.Core.Serialization.ObjectToInferredTypesConverter)
+                            options.Converters.RemoveAt(i);
+                    }
+                    options.Converters.Insert(0, new Exceptionless.Core.Serialization.ObjectToInferredTypesConverter(preferInt64: true));
+                }));


-        settings.Formatting = Formatting.Indented;
+        // Use snake_case fixture — matches production ES format (all data stored as snake_case).
+        // The original 1477.expected.json has PascalCase keys for the upgrader test.
+        string json = File.ReadAllText(Path.GetFullPath(Path.Combine("..", "..", "..", "Plugins", "WebHookData", "1477.snake-case.json")));


+
+        var targetInfo = error.Data?.GetValue<SettingsDictionary>(Error.KnownDataKeys.TargetInfo, _serializer);
+        Assert.NotNull(targetInfo);
+        Assert.True(targetInfo.ContainsKey("ExceptionType"), "@target should contain ExceptionType");


+        Assert.NotNull(targetInfo);
+        Assert.True(targetInfo.ContainsKey("ExceptionType"), "@target should contain ExceptionType");
+        Assert.Equal("System.InvalidOperationException", targetInfo["ExceptionType"]);
+        Assert.True(targetInfo.ContainsKey("Method"), "@target should contain Method");


github-actions · 2026-05-13T20:13:29Z

Package	Line Rate	Branch Rate	Complexity	Health
Exceptionless.Core	64%	58%	7702	➖
Exceptionless.Insulation	24%	23%	203	❌
Exceptionless.Web	60%	46%	3916	➖
Exceptionless.AppHost	19%	15%	88	❌
Summary	61% (12331 / 20241)	53% (6083 / 11452)	11909	➖

github-code-quality Bot found potential problems Feb 28, 2026

View reviewed changes

niemyjski requested a review from Copilot February 28, 2026 23:10

niemyjski self-assigned this Feb 28, 2026

Copilot started reviewing on behalf of niemyjski February 28, 2026 23:10 View session

Copilot AI reviewed Feb 28, 2026

View reviewed changes

Comment thread src/Exceptionless.Core/Serialization/ElasticSystemTextJsonSerializer.cs Outdated

Comment thread src/Exceptionless.Core/Serialization/ElasticSystemTextJsonSerializer.cs Outdated

niemyjski requested a review from Copilot February 28, 2026 23:51

Copilot started reviewing on behalf of niemyjski February 28, 2026 23:51 View session

Copilot AI reviewed Feb 28, 2026

View reviewed changes

Comment thread src/Exceptionless.Core/Serialization/ElasticSystemTextJsonSerializer.cs Outdated

Comment thread src/Exceptionless.Core/Serialization/ObjectToInferredTypesConverter.cs Outdated

github-code-quality Bot found potential problems Mar 1, 2026

View reviewed changes

niemyjski force-pushed the feature/system-text-json-v2 branch from e6a3b35 to c292e13 Compare March 1, 2026 00:52

github-code-quality Bot found potential problems Mar 1, 2026

View reviewed changes

niemyjski force-pushed the feature/system-text-json-v2 branch 2 times, most recently from 4d880ad to 09134ce Compare March 1, 2026 01:19

github-code-quality Bot found potential problems Mar 1, 2026

View reviewed changes

Comment thread src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs Fixed

Comment thread src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs Fixed

niemyjski and others added 2 commits March 21, 2026 19:50

Copilot AI review requested due to automatic review settings March 22, 2026 00:51

niemyjski force-pushed the feature/system-text-json-v2 branch from 09134ce to 13aa277 Compare March 22, 2026 00:51

Copilot started reviewing on behalf of niemyjski March 22, 2026 00:51 View session

Copilot AI reviewed Mar 22, 2026

View reviewed changes

Comment thread tests/Exceptionless.Tests/IntegrationTestsBase.cs Outdated

Comment thread tests/Exceptionless.Tests/Exceptionless.Tests.csproj

Comment thread src/Exceptionless.Core/Exceptionless.Core.csproj Outdated

Comment thread src/Exceptionless.Core/Utility/TypeHelper.cs Outdated

niemyjski added 4 commits March 22, 2026 11:11

Migrate index configurations to new Elasticsearch 8 fluent API

9f7914b

Update all index files to use CreateIndexRequestDescriptor, void returns for ConfigureIndex/ConfigureIndexMapping, expression-based property mappings, DynamicMapping enum, and renamed analysis methods.

Migrate queries and repositories to Elastic.Clients.Elasticsearch

4559885

Replace QueryContainer with Query, use TermQuery/TermsQuery/BoolQuery/DateRangeQuery object initializers with Infer.Field expressions.

Migrate jobs, migrations, and controllers to Elastic.Clients.Elastics…

6be3932

…earch Update CleanupOrphanedDataJob, DataMigrationJob, all migration files, AdminController, EventController, ProjectController, and test files to use new ES 8 client APIs.

niemyjski requested a review from Copilot March 22, 2026 16:12

Copilot started reviewing on behalf of niemyjski March 22, 2026 16:13 View session

Copilot AI reviewed Mar 22, 2026

View reviewed changes

Comment thread src/Exceptionless.Core/Repositories/OrganizationRepository.cs

github-code-quality Bot found potential problems Mar 22, 2026

View reviewed changes

Comment thread src/Exceptionless.Core/Jobs/Elastic/DataMigrationJob.cs Fixed

Comment thread src/Exceptionless.Core/Repositories/Configuration/ExceptionlessElasticConfiguration.cs Fixed

niemyjski added 2 commits March 22, 2026 11:19

fix: remove duplicate Id mapping in EventIndex

3890cb4

SetupDefaults() already maps the Id property. The explicit Keyword(e => e.Id) mapping caused a duplicate key error in the new Elastic.Clients.Elasticsearch client.

niemyjski commented May 13, 2026

View reviewed changes

github-code-quality Bot found potential problems May 13, 2026

View reviewed changes

		@@ -43,6 +43,25 @@ namespace Exceptionless.Core.Serialization;
		/// <seealso href="https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/converters-how-to#deserialize-inferred-types-to-object-properties"/>
		public sealed class ObjectToInferredTypesConverter : JsonConverter<object?>

		Assert.NotNull(snapshots.Repositories);
		Assert.NotNull(snapshots.Snapshots);

Uh oh!

Conversation

niemyjski commented Feb 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!