-
Notifications
You must be signed in to change notification settings - Fork 0
feat: raster layer metadata schema #123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vitaligi
wants to merge
11
commits into
master
Choose a base branch
from
feat/layer-metadata-schema
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
ce9b6d5
feat: raster layer metadata schema
vitaligi 93a4413
refactor: import type
vitaligi c291edc
refactor: stricter validation for raster layer metadata
vitaligi b100ce9
fix: duplicate id property
vitaligi 9d2240c
refactor: rename file schema and type
vitaligi a7e0bbf
refactor: modify optional properties based on db schema
vitaligi 27bd4a1
refactor: move type to other file
vitaligi 74c2ac5
fix: handle case where minHorizontalAccuracyCE90 is undefined
vitaligi 182fcef
fix: productStatus is not optional
vitaligi 98ffe37
fix: link optional properties
vitaligi c5b1399
fix: incorrect minResolutionMeter schema
vitaligi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| import { RecordStatus, RecordType } from '@map-colonies/types'; | ||
| import z from 'zod'; | ||
| import { CORE_VALIDATIONS, INGESTION_VALIDATIONS, TileOutputFormat, Transparency } from '../../constants'; | ||
| import { multiPolygonSchema, polygonSchema } from './geo.schema'; | ||
| import { rasterProductTypeSchema, resourceIdSchema, versionSchema } from './job.schema'; | ||
| import { tilesMimeFormatSchema } from './mime.schema'; | ||
|
|
||
| export const rasterLayerCatalogSchema = z | ||
| .object({ | ||
| metadata: z | ||
| .object({ | ||
| id: z.string().uuid(), | ||
| srs: z.literal('4326'), | ||
| productVersion: versionSchema, | ||
| maxResolutionDeg: z | ||
| .number({ message: 'Max resolution degree should be a number' }) | ||
| .min(CORE_VALIDATIONS.resolutionDeg.min, { | ||
| message: `Max resolution degree should not be less than ${CORE_VALIDATIONS.resolutionDeg.min}`, | ||
| }) | ||
| .max(CORE_VALIDATIONS.resolutionDeg.max, { | ||
| message: `Max resolution degree should not be larger than ${CORE_VALIDATIONS.resolutionDeg.max}`, | ||
| }), | ||
| minResolutionDeg: z | ||
| .number({ message: 'Min resolution degree should be a number' }) | ||
| .min(CORE_VALIDATIONS.resolutionDeg.min, { | ||
| message: `Min resolution degree should not be less than ${CORE_VALIDATIONS.resolutionDeg.min}`, | ||
| }) | ||
| .max(CORE_VALIDATIONS.resolutionDeg.max, { | ||
| message: `Min resolution degree should not be larger than ${CORE_VALIDATIONS.resolutionDeg.max}`, | ||
| }), | ||
| scale: z.number().min(INGESTION_VALIDATIONS.scale.min).max(INGESTION_VALIDATIONS.scale.max).optional(), | ||
| creationDateUTC: z.coerce.date(), | ||
| ingestionDate: z.coerce.date(), | ||
| minHorizontalAccuracyCE90: z | ||
| .number({ message: 'Min horizontal accuracy CE90 should be a number' }) | ||
| .min(INGESTION_VALIDATIONS.horizontalAccuracyCE90.min, { | ||
| message: `Min horizontal accuracy CE90 should not be less than ${INGESTION_VALIDATIONS.horizontalAccuracyCE90.min}`, | ||
| }) | ||
| .max(INGESTION_VALIDATIONS.horizontalAccuracyCE90.max, { | ||
| message: `Min horizontal accuracy CE90 should not be larger than ${INGESTION_VALIDATIONS.horizontalAccuracyCE90.max}`, | ||
| }) | ||
| .optional(), | ||
| maxHorizontalAccuracyCE90: z | ||
| .number({ message: 'Max horizontal accuracy CE90 should be a number' }) | ||
| .min(INGESTION_VALIDATIONS.horizontalAccuracyCE90.min, { | ||
| message: `Max horizontal accuracy CE90 should not be less than ${INGESTION_VALIDATIONS.horizontalAccuracyCE90.min}`, | ||
| }) | ||
| .max(INGESTION_VALIDATIONS.horizontalAccuracyCE90.max, { | ||
| message: `Max horizontal accuracy CE90 should not be larger than ${INGESTION_VALIDATIONS.horizontalAccuracyCE90.max}`, | ||
| }), | ||
| region: z.array(z.string().min(1)).min(1), | ||
| sensors: z | ||
| .array( | ||
| z.string({ message: 'Sensors should be an array of strings' }).regex(new RegExp(INGESTION_VALIDATIONS.sensor.pattern), { | ||
| message: 'Sensors should be an array with items not starting or ending with whitespace characters', | ||
| }), | ||
| { message: 'Sensors should be an array' } | ||
| ) | ||
| .min(1, { message: 'Sensors should have an array length of at least 1' }), | ||
| imagingTimeBeginUTC: z.coerce.date({ message: 'Imaging time begin UTC should be a datetime' }), | ||
| imagingTimeEndUTC: z.coerce.date({ message: 'Imaging time end UTC should be a datetime' }), | ||
| updateDateUTC: z.coerce.date(), | ||
| maxResolutionMeter: z | ||
| .number({ message: 'Max resolution meter should be a number' }) | ||
| .min(INGESTION_VALIDATIONS.resolutionMeter.min, { | ||
| message: `Max resolution meter should not be less than ${INGESTION_VALIDATIONS.resolutionMeter.min}`, | ||
| }) | ||
| .max(INGESTION_VALIDATIONS.resolutionMeter.max, { | ||
| message: `Max resolution meter should not be larger than ${INGESTION_VALIDATIONS.resolutionMeter.max}`, | ||
| }), | ||
| minResolutionMeter: z | ||
| .number({ message: 'Min resolution meter should be a number' }) | ||
| .min(INGESTION_VALIDATIONS.resolutionMeter.min, { | ||
| message: `Min resolution meter should not be less than ${INGESTION_VALIDATIONS.resolutionMeter.min}`, | ||
| }) | ||
| .max(INGESTION_VALIDATIONS.resolutionMeter.max, { | ||
| message: `Min resolution meter should not be larger than ${INGESTION_VALIDATIONS.resolutionMeter.max}`, | ||
| }), | ||
| productSubType: z.string().optional(), | ||
| productBoundingBox: z | ||
| .string({ message: 'Product bounding box should be a string' }) | ||
| .regex(new RegExp(INGESTION_VALIDATIONS.boundingBox.pattern), { | ||
| message: 'Product bounding box must be of the shape min_x,min_y,max_x,max_y', | ||
| }) | ||
| .optional(), | ||
| displayPath: z.string().uuid(), | ||
| transparency: z.nativeEnum(Transparency), | ||
| tileMimeFormat: tilesMimeFormatSchema, | ||
| tileOutputFormat: z.nativeEnum(TileOutputFormat), | ||
| productStatus: z.nativeEnum(RecordStatus), | ||
| type: z.literal(RecordType.RECORD_RASTER), | ||
| classification: z | ||
| .string() | ||
| .regex(new RegExp(INGESTION_VALIDATIONS.classification.pattern), { message: 'Classification value must be between 0 and 100' }), | ||
| productName: z.string().optional(), | ||
| description: z.string().optional(), | ||
| srsName: z.literal('WGS84GEO'), | ||
| producerName: z.string().optional(), | ||
| footprint: polygonSchema.or(multiPolygonSchema), | ||
| productId: resourceIdSchema, | ||
| productType: rasterProductTypeSchema, | ||
| }) | ||
| .strict() | ||
| .refine( | ||
| (rasterLayerMetadata) => | ||
| rasterLayerMetadata.imagingTimeBeginUTC <= rasterLayerMetadata.imagingTimeEndUTC && rasterLayerMetadata.imagingTimeEndUTC <= new Date(), | ||
| { | ||
| message: 'Imaging time begin UTC should be less than or equal to imaging time end UTC and both less than or equal to current timestamp', | ||
| } | ||
| ) | ||
| .refine( | ||
| (rasterLayerMetadata) => | ||
| rasterLayerMetadata.minHorizontalAccuracyCE90 !== undefined | ||
| ? rasterLayerMetadata.maxHorizontalAccuracyCE90 <= rasterLayerMetadata.minHorizontalAccuracyCE90 | ||
| : true, | ||
| { | ||
| message: 'Max horizontal accuracy CE90 should be less than or equal to min horizontal accuracy CE90', | ||
| } | ||
| ) | ||
| .refine((rasterLayerMetadata) => rasterLayerMetadata.maxResolutionDeg <= rasterLayerMetadata.minResolutionDeg, { | ||
| message: 'Max resolution degree should be less than or equal to min resolution degree', | ||
| }) | ||
| .refine((rasterLayerMetadata) => rasterLayerMetadata.maxResolutionMeter <= rasterLayerMetadata.minResolutionMeter, { | ||
| message: 'Max resolution meter should be less than or equal to min resolution meter', | ||
| }), | ||
| links: z | ||
| .array( | ||
| z | ||
| .object({ | ||
| protocol: z.string(), | ||
| url: z.string().url(), | ||
| name: z.string().optional(), | ||
| description: z.string().optional(), | ||
| }) | ||
| .strict() | ||
| ) | ||
| .optional(), | ||
| }) | ||
| .strict() | ||
| .describe('rasterLayerCatalogSchema'); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,34 +1,4 @@ | ||
| import z from 'zod'; | ||
| import { IMetadataCommonModel, RecordStatus, TilesMimeFormat } from '@map-colonies/types'; | ||
| import { TileOutputFormat, Transparency } from '../../constants/core'; | ||
| import { aggregationFeatureSchema } from '../../schemas'; | ||
|
|
||
| // eslint-disable-next-line @typescript-eslint/consistent-type-definitions | ||
| export type RasterLayerMetadata = { | ||
| id: string; | ||
| srs: string; | ||
| productVersion: string; | ||
| maxResolutionDeg: number; | ||
| minResolutionDeg: number; | ||
| scale?: number; | ||
| creationDateUTC?: Date; | ||
| ingestionDate?: Date; | ||
| minHorizontalAccuracyCE90: number; | ||
| maxHorizontalAccuracyCE90: number; | ||
| region: string[]; | ||
| sensors: string[]; | ||
| imagingTimeBeginUTC: Date; | ||
| imagingTimeEndUTC: Date; | ||
| updateDateUTC?: Date; | ||
| maxResolutionMeter: number; | ||
| minResolutionMeter: number; | ||
| productSubType?: string; | ||
| productBoundingBox?: string; | ||
| displayPath: string; | ||
| transparency: Transparency; | ||
| tileMimeFormat: TilesMimeFormat; | ||
| tileOutputFormat: TileOutputFormat; | ||
| productStatus?: RecordStatus; | ||
| } & IMetadataCommonModel; | ||
| import type { aggregationFeatureSchema } from '../../schemas'; | ||
|
|
||
| export type AggregationFeature = z.infer<typeof aggregationFeatureSchema>; |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The type you changed is currently used in
mc-modelsand is also used by the app team. Could you please verify that your new type doesn't break anything in those places? It would be good to check:mc-models