Commit Graph

139 Commits

Author SHA1 Message Date
Nicolas
a8d6af93aa restore: add Kubernetes metadata to JSON logs
- Restore ParseJSON operations to add Kubernetes metadata to log body
- Add kubernetes.pod, kubernetes.namespace, kubernetes.pod_ip, etc. to JSON
- This will enrich the JSON logs with Kubernetes context information
- Maintain original JSON structure while adding K8s metadata
2025-09-22 21:46:38 +08:00
Nicolas
2983250d07 fix: remove json_parser to preserve original JSON format
- Remove json_parser operator to keep logs in original JSON format
- This should maintain the structured JSON content in Grafana
- Logs will be sent as raw JSON without parsing into attributes
2025-09-22 21:45:14 +08:00
Nicolas
cda371e74f fix: change start_at to beginning to read existing logs
- Change start_at from end to beginning
- This should allow OpenTelemetry Collector to read existing log content
- May cause duplicate logs but should make logs visible in Grafana
2025-09-22 21:38:59 +08:00
Nicolas
620bacdbbf fix: remove move operator that may be causing log processing failure
- Remove move operator from json_parser configuration
- Keep only json_parser with parse_to: attributes
- This should allow logs to be processed correctly
2025-09-22 21:38:00 +08:00
Nicolas
802698bcb7 fix: align authentication logging with chat service configuration
- Add parse_to: attributes to json_parser operator
- Add move operator to move text content to body
- Use application instead of service_name for consistency
- Use global.environment for environment setting
- Add global.environment to values files for both alpha and prod
- This should fix the unknown_service classification issue
2025-09-22 21:21:19 +08:00
Nicolas
fe2b7200de simplify: remove where clause from transform processor
- Remove complex where clause that may cause syntax errors
- Set environment to alpha by default for all logs
- This should fix the transform processor execution issue
2025-09-22 20:45:02 +08:00
Nicolas
c4719493af fix: add service.name attribute to prevent unknown_service classification
- Add service.name resource attribute in addition to service_name
- This should prevent logs from being classified as unknown_service in Loki
- Keep both attributes for compatibility
2025-09-22 20:40:30 +08:00
Nicolas
b1554522b5 Fix the configuration of the JSON parser 2025-09-22 20:35:06 +08:00
Nicolas
b60c5c9324 simplify: remove ParseJSON operations from OpenTelemetry transform
- Remove complex JSON parsing operations that may cause issues
- Keep only essential resource attribute mapping
- This should help logs be properly processed and sent to Loki
2025-09-22 20:22:40 +08:00
Nicolas
a9d3930ba3 fix: use where clause for conditional environment setting
- Replace if statement with where clause for environment mapping
- Set default to alpha, then override to prod for freeleaps-prod namespace
2025-09-22 19:54:57 +08:00
Nicolas
0774325bf6 fix: use if statement instead of ReplaceAll for environment mapping
- Replace ReplaceAll function with if statement for environment mapping
- This avoids undefined function error in OpenTelemetry Collector OTTL
2025-09-22 19:53:13 +08:00
Nicolas
a9d6ab93e6 fix the bug of auth 2025-09-22 19:44:52 +08:00
Nicolas
a81b061d31 update authentication 2025-09-22 19:40:17 +08:00
Nicolas
b1f6ce8a1d Set the log environment for authentication using string replacement 2025-09-22 17:47:40 +08:00
Nicolas
61509dabf2 roll back 2025-09-22 17:33:22 +08:00
Nicolas
d9209e89ef fix authentication 2025-09-22 17:31:11 +08:00
Nicolas
5b185972a8 update authentication log set 2025-09-22 16:53:54 +08:00
Nicolas
efa0b55236 change authentication log env set 2025-09-22 16:23:01 +08:00
1f993da23e revert e76da4ea18
revert Remove all such cert usage(across all services) if service is already deployed in k8s
2025-08-29 03:53:51 +00:00
Nicolas
e76da4ea18 Remove all such cert usage(across all services) if service is already deployed in k8s 2025-08-29 10:03:35 +08:00
Nicolas
86f989e405 The production environment of the authentication service also fully supports AKV. 2025-08-18 17:44:17 +08:00
zhenyus
363b408d39 refactor(authentication): update key reference in FreeleapsSecret template
- Changed the key reference from 'key' to 'secretKey' in freeleapssecret.yaml to improve clarity and consistency in secret management.
- This adjustment aligns with the recent updates to the secret management structure within the authentication module.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-08-18 15:33:28 +08:00
zhenyus
05aa22215e refactor(authentication): streamline secret reference assignment in deployment.yaml
- Introduced a variable for the target secret name to improve readability and maintainability of the secret injection logic.
- This change enhances the clarity of the deployment configuration for the authentication module.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-08-18 15:29:00 +08:00
zhenyus
e34a57bef5 refactor(authentication): update secret references in deployment and FreeleapsSecret templates
- Changed references from .Values.secrets to .Values.authentication.secrets in deployment.yaml and freeleapssecret.yaml to align with the new secret management structure.
- This update enhances the clarity and organization of secret management within the authentication module.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-08-18 15:27:26 +08:00
zhenyus
4da0d64995 feat(authentication): integrate secret management for JWT and MongoDB URI
- Added a secrets section in values.alpha.yaml to reference JWT secret key and MongoDB URI from a FreeleapsSecretStore.
- Updated deployment.yaml to inject these secrets as environment variables, enhancing security and configuration management.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-08-18 15:25:48 +08:00
zhenyus
1af1073eab fix(authentication): update VPA deployment name to 'authentication'
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-06-10 16:05:25 +08:00
zhenyus
db590f3f27 refactor: update gitea-webhook-ambassador Dockerfile and configuration
- Changed the build process to include a web UI build stage using Node.js.
- Updated Go build stage to copy web UI files to the correct location.
- Removed the main.go file as it is no longer needed.
- Added SQLite database configuration to example config.
- Updated dependencies in go.mod and go.sum, including new packages for JWT and SQLite.
- Modified .gitignore to include new database and configuration files.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-06-10 16:00:52 +08:00
zhenyus
6d9d15d4d2 Add OpenTelemetry logging support across multiple services
- Introduced log ingestion configuration in values files for centralStorage, content, notification, and payment services.
- Updated deployment templates to conditionally include OpenTelemetry annotations and volume mounts based on log ingestion settings.
- Added OpenTelemetry RBAC configurations for service accounts and cluster roles to enable logging.
- Implemented OpenTelemetry collector configuration to process logs and export them to Loki.
- Ensured compatibility with existing Helm chart structure and maintained backward compatibility for services without log ingestion enabled.

Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 22:03:00 +08:00
zhenyus
84e4eb9f02 fix(opentelemetry): update logging level to info and remove debug exporter
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 20:11:45 +08:00
zhenyus
bd31bcda9d fix(opentelemetry): simplify log body assignment by removing unnecessary type conversion
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 20:06:59 +08:00
zhenyus
dc3e06738c fix(opentelemetry): update log body processing to directly use resource attributes
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 20:05:40 +08:00
zhenyus
ef0cf0fa87 fix(opentelemetry): add batch configuration for log processing
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 20:03:10 +08:00
zhenyus
ebeea26864 fix(opentelemetry): enhance log body processing by restructuring Kubernetes attributes handling
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 20:01:45 +08:00
zhenyus
6588193c7d fix(opentelemetry): reorder processors in log pipeline for correct processing sequence
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 19:53:40 +08:00
zhenyus
32d9f41ee5 fix(opentelemetry): update processor attributes to use environment variable values for improved metadata extraction
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 19:48:24 +08:00
zhenyus
ddabde9dc0 fix(opentelemetry): update environment variables and resource attributes for improved metadata extraction
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 19:43:22 +08:00
zhenyus
15dd1fba0b fix(opentelemetry): update resource attributes in distributor and log transformation for improved metadata extraction
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:43:08 +08:00
zhenyus
5c2ea75928 fix(opentelemetry): correct sources key in pod association configuration
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:29:14 +08:00
zhenyus
9ceaaa3e0b fix(opentelemetry): restore and configure k8sattributes processor for enhanced metadata extraction
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:28:25 +08:00
zhenyus
c08ddf7f76 fix(opentelemetry): add resource attributes for application and environment in log transformation
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:20:39 +08:00
zhenyus
473f5cea54 fix(opentelemetry): update log transformation to use resource attributes for application and environment
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:17:40 +08:00
zhenyus
802b9bf8ba fix(opentelemetry): correct attribute path for service and deployment in log transformation
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:07:42 +08:00
zhenyus
c106c9a624 fix(loki): update resource_attributes to use regex for indexing labels
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 17:05:53 +08:00
zhenyus
f45f79e32e fix(opentelemetry): update processors in logs pipeline to use transform instead of attributes
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:44:36 +08:00
zhenyus
fb368eb8d8 fix(opentelemetry): remove commented-out k8sattributes actions for clarity
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:44:13 +08:00
zhenyus
b0a1e56a75 fix(opentelemetry): correct syntax for accessing context attributes in log transformation
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:43:24 +08:00
zhenyus
5f1742e22e refactor(opentelemetry): reorganize log transformation logic for clarity and consistency
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:42:28 +08:00
zhenyus
2e1879298f fix(opentelemetry): remove k8sattributes processor from logs pipeline for simplification
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:32:06 +08:00
zhenyus
1967a7444c fix(opentelemetry): comment out unused k8sattributes processor and related actions for clarity
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:31:29 +08:00
zhenyus
80e75eea27 fix(opentelemetry): add telemetry logging level configuration for improved log management
Signed-off-by: zhenyus <zhenyus@mathmast.com>
2025-04-21 16:26:06 +08:00