18 KiB
18 KiB
Reconciler Architecture Deep Dive
🎯 Overview
Your freeleaps-devops-reconciler is a sophisticated Kubernetes Operator that orchestrates your entire DevOps infrastructure. It's not just a simple CRD controller - it's a full-stack DevOps automation platform that bridges your Git repositories, container registries, Jenkins pipelines, ArgoCD applications, and Kubernetes deployments.
🏗️ Architecture Overview
🔄 The Big Picture: How Your Reconciler Works
┌─────────────────────────────────────────────────────────────────────────────┐
│ FRELEAPS DEVOPS RECONCILER ARCHITECTURE │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ KUBERNETES OPERATOR (KOPF) │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ DevOpsProject │ │ ArgoSetting │ │ JenkinsSetting│ │ IngressResource│ │
│ │ │ Controller │ │ Controller │ │ Controller │ │ Controller │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ EXTERNAL SERVICE INTEGRATION │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ ArgoCD │ │ Jenkins │ │ Docker Hub │ │ GoDaddy │ │
│ │ │ Client │ │ Client │ │ Client │ │ Client │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ MESSAGING & EVENT SYSTEM │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ │ RabbitMQ │ │ Heartbeat │ │ Deployment │ │ TTL Monitor │ │
│ │ │ Listener │ │ Sender │ │ Monitor │ │ Manager │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
🔧 Core Components Deep Dive
1. DevOpsProject Controller 🏗️
What it does: The orchestrator-in-chief that creates your entire DevOps ecosystem.
Real Code Example:
@kopf.on.create(group=consts.GROUP, version=consts.VERSION, kind=consts.DEVOPS_PROJECT_KIND)
def on_devops_proj_created(name: str, namespace: Optional[str], body: Body, logger: Logger, **kwargs):
# When you create a DevOpsProject CR, this triggers:
# 1. Validates your Git repo and container registry config
# 2. Creates ArgoSetting CR (for ArgoCD management)
# 3. Creates JenkinsSetting CR (for CI/CD pipelines)
# 4. Creates ContainerRegistry CR (for image management)
# 5. Creates GitCredentials CR (for authentication)
Your Actual Flow:
User creates DevOpsProject CR
↓
Reconciler validates Git repo + container registry
↓
Creates ArgoSetting CR (manages ArgoCD projects/apps)
↓
Creates JenkinsSetting CR (manages CI/CD pipelines)
↓
Creates ContainerRegistry CR (manages Docker images)
↓
Creates GitCredentials CR (manages authentication)
↓
Your DevOps ecosystem is ready! 🎉
2. ArgoSetting Controller 🚀
What it does: Manages your ArgoCD infrastructure - projects, repositories, and applications.
Real Code Example:
# When ArgoSetting CR is created:
for project in as_spec.projects:
# Creates ArgoCD Project
desired_resources.append(ManagedResource(
resource_type="project",
resource_id=project.name,
description=project.desc,
metadata={
"source_repos": [repo.url for repo in as_spec.repositories],
"destinations": [{"server": dest.server, "namespace": dest.namespace}]
}
))
for app in as_spec.applications:
# Creates ArgoCD Application
desired_resources.append(ManagedResource(
resource_type="application",
resource_id=app.name,
metadata={
"project": app.project,
"repo_url": app.source.repo_url,
"path": app.source.path,
"target_revision": app.source.revision
}
))
Your Actual ArgoCD Management:
ArgoSetting CR created
↓
Reconciler connects to ArgoCD API (argo.mathmast.com)
↓
Creates ArgoCD Project (defines permissions, repos, destinations)
↓
Creates ArgoCD Repository (connects to your Git repo)
↓
Creates ArgoCD Application (deploys your app)
↓
ArgoCD starts syncing your application! 🔄
3. JenkinsSetting Controller ⚙️
What it does: Manages your Jenkins CI/CD pipelines - creates folders, pipelines, and credentials.
Real Code Example:
@kopf.timer(group=consts.GROUP, version=consts.VERSION, kind=consts.JENKINS_SETTINGS_KIND, interval=300)
def poll_project_config(name: str, namespace: str, body: Body, logger: logging.Logger, **kwargs):
# Every 5 minutes, the reconciler:
# 1. Fetches your project's YAML config from Git
# 2. Generates Jenkins Pipeline DSL
# 3. Creates/updates Jenkins pipelines
# 4. Manages pipeline credentials
Your Actual Jenkins Management:
JenkinsSetting CR created
↓
Reconciler clones your Git repo
↓
Reads your project's YAML configuration
↓
Generates Jenkins Pipeline DSL (Groovy script)
↓
Creates Jenkins folder structure (project/environment)
↓
Creates Jenkins pipeline with your DSL
↓
Your CI/CD pipeline is ready! 🚀
4. DeploymentRecord Controller 🎯
What it does: Orchestrates actual deployments - triggers Jenkins builds, monitors ArgoCD sync, manages TTL.
Real Code Example:
@kopf.on.create(group=consts.GROUP, version=consts.VERSION, plural=consts.DEPLOYMENT_RECORD_PLURAL)
async def create_deployment_record(spec: Dict[str, Any], name: str, namespace: str, uid: str, logger: Logger, **kwargs):
# When you trigger a deployment:
# 1. Validates deployment request
# 2. Triggers Jenkins build
# 3. Monitors build progress
# 4. Triggers ArgoCD sync
# 5. Monitors deployment status
# 6. Manages TTL (Time To Live)
Your Actual Deployment Flow:
User clicks "Deploy" button
↓
DeploymentRecord CR created
↓
Reconciler triggers Jenkins build
↓
Monitors build phases (building, testing, packaging)
↓
Triggers ArgoCD sync when build completes
↓
Monitors ArgoCD sync status
↓
Creates IngressResource for external access
↓
Deployment is live! 🌐
5. IngressResource Controller 🌐
What it does: Manages external access - DNS records, SSL certificates, and ingress rules.
Real Code Example:
async def create_ingress_resource(self, body: Body, name: str, namespace: str, **kwargs):
# When IngressResource CR is created:
# 1. Creates DNS record via GoDaddy API
# 2. Requests SSL certificate via cert-manager
# 3. Creates Kubernetes Ingress
# 4. Updates deployment URL
# 5. Sends heartbeat with live URL
Your Actual Ingress Management:
IngressResource CR created
↓
Reconciler calls GoDaddy API
↓
Creates DNS record (app.mathmast.com → 4.155.160.32)
↓
Requests SSL certificate from Let's Encrypt
↓
Creates Kubernetes Ingress rule
↓
Your app is accessible at https://app.mathmast.com! 🔒
📡 Messaging & Event System
RabbitMQ Integration 🐰
Your reconciler uses RabbitMQ for asynchronous communication and event-driven architecture.
Event Types:
class EventType(Enum):
DEVOPS_INITIALIZE = "DevOpsInitialize" # New project setup
DEVOPS_RECONCILE = "DevOpsReconcile" # Deployment trigger
DEVOPS_RECONCILE_HEARTBEAT = "DevOpsReconcileJobHeartbeat" # Progress updates
Real Event Flow:
User triggers deployment
↓
DevOpsReconcileEvent sent to RabbitMQ
↓
Reconciler picks up event
↓
Creates DeploymentRecord CR
↓
Sends heartbeat every 30 seconds
↓
User sees real-time progress! 📊
Heartbeat System 💓
What it does: Provides real-time deployment status to your users.
Real Code Example:
@dataclass
class DevOpsReconcileJobHeartbeatEvent:
operation: str = "heartbeat"
id: str = "" # deployment ID
status: str = "running" # running/success/failed/terminated
phase: str = "initializing" # current deployment phase
phase_message: str = "" # human-readable message
url: Optional[str] = None # live URL when deployment completes
Your Actual Heartbeat Flow:
Deployment starts
↓
Heartbeat every 30 seconds:
- Phase: "initializing" → "building" → "deploying" → "verifying"
- Status: "running" → "success"
- URL: None → "https://app.mathmast.com"
↓
User sees live progress in UI! 📈
🕒 TTL (Time To Live) Management
Automatic Cleanup 🧹
Your reconciler includes sophisticated TTL management for temporary deployments.
Real Code Example:
@dataclass
class TTLMonitoringState:
deployment_record_name: str
enabled: bool
ttl_seconds: int # Default: 3 hours (10800 seconds)
start_time: datetime
expiration_time: datetime
phase: TTLMonitoringPhase # monitoring/cleanup/completed
Your Actual TTL Flow:
Deployment completes
↓
TTL monitoring starts (3 hours by default)
↓
Every minute: Check if TTL expired
↓
When expired: Trigger cleanup
↓
Delete ArgoCD applications
↓
Delete Kubernetes resources
↓
Delete DNS records
↓
Delete SSL certificates
↓
Environment cleaned up! 🧹
🔗 Relationship with Your DevOps Infrastructure
How the Reconciler Extends Kubernetes 🔧
Your reconciler doesn't just manage Kubernetes resources - it extends Kubernetes with custom DevOps capabilities:
┌─────────────────────────────────────────────────────────────────────────────┐
│ KUBERNETES API EXTENSION │
│ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ CUSTOM RESOURCES (CRs) │ │
│ │ │ │
│ │ DevOpsProject CR → ArgoSetting CR → JenkinsSetting CR │ │
│ │ ↓ ↓ ↓ │ │
│ │ ContainerRegistry CR → GitCredentials CR → IngressResource CR │ │
│ │ ↓ ↓ ↓ │ │
│ │ DeploymentRecord CR → TTL Management → Heartbeat System │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ EXTERNAL SERVICE ORCHESTRATION │ │
│ │ │ │
│ │ ArgoCD API → Jenkins API → Docker Hub API → GoDaddy API │ │
│ │ ↓ ↓ ↓ ↓ │ │
│ │ Git Repos → CI/CD Pipelines → Container Images → DNS Records │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
Your Actual DevOps Workflow 🔄
Complete Flow Example:
1. Developer pushes code to Git
↓
2. User creates DevOpsProject CR
↓
3. Reconciler creates ArgoSetting CR
↓
4. Reconciler creates JenkinsSetting CR
↓
5. Jenkins pipeline created and triggered
↓
6. Build completes, image pushed to registry
↓
7. ArgoCD syncs new image
↓
8. IngressResource creates external access
↓
9. App is live at https://app.mathmast.com
↓
10. TTL monitoring starts (3 hours)
↓
11. After 3 hours: Automatic cleanup
🎯 Key Benefits of Your Reconciler Architecture
1. Declarative DevOps 📝
- Define once, deploy everywhere: Your DevOpsProject CR defines your entire infrastructure
- GitOps workflow: Everything is version-controlled and declarative
- Consistency: Same process for alpha and production environments
2. Automation at Scale 🤖
- Zero manual intervention: From Git push to live deployment
- Multi-environment support: Alpha and production with same configuration
- Automatic cleanup: TTL management prevents resource waste
3. Real-time Visibility 👁️
- Live progress tracking: Heartbeat system shows real-time deployment status
- Comprehensive monitoring: Every phase is tracked and reported
- Error handling: Detailed error messages and recovery mechanisms
4. Enterprise Integration 🏢
- Multi-service orchestration: ArgoCD, Jenkins, Docker Hub, GoDaddy
- Security: Credential management and SSL certificate automation
- Scalability: Kubernetes-native architecture scales with your cluster
🔍 Your Reconciler vs. Traditional DevOps
Traditional DevOps 🏗️
Manual Jenkins setup → Manual ArgoCD config → Manual DNS setup → Manual SSL setup
Your Reconciler 🚀
DevOpsProject CR → Automatic Jenkins + ArgoCD + DNS + SSL setup
The difference: Your reconciler transforms manual DevOps tasks into declarative, automated, and scalable operations that run on Kubernetes.
🎉 Conclusion
Your freeleaps-devops-reconciler is not just a Kubernetes operator - it's a complete DevOps automation platform that:
- Extends Kubernetes with custom DevOps capabilities
- Orchestrates multiple external services (ArgoCD, Jenkins, Docker Hub, GoDaddy)
- Provides real-time visibility into deployment progress
- Automates complex workflows from Git push to live deployment
- Manages the complete lifecycle including cleanup and TTL
It's the brain of your DevOps infrastructure, making complex multi-service orchestration as simple as creating a Kubernetes Custom Resource! 🧠✨