Database backups (#75)
* initial commit * test fix * manual test * copilot comments * code quality skill * comments * fix types problem * formatting * tests now correct for new backup method * UTC dates
This commit is contained in:
Shaheer Sarfaraz
GitHub
parent
2841a7fe3a
commit
179deffe13
145
.opencode/skills/code-quality/SKILL.md
Normal file
145
.opencode/skills/code-quality/SKILL.md
Normal file
@ -0,0 +1,145 @@
|
|||||||
|
---
|
||||||
|
name: orchestrator-code-quality
|
||||||
|
description: Generate a code quality report for the orchestrator/ folder focusing on duplication, complexity hotspots, and error/logging consistency, using npm run check:all as the primary gate; report only (no code changes).
|
||||||
|
license: MIT
|
||||||
|
compatibility: opencode
|
||||||
|
metadata:
|
||||||
|
audience: maintainers
|
||||||
|
repo_area: orchestrator
|
||||||
|
primary_command: npm run check:all
|
||||||
|
---
|
||||||
|
|
||||||
|
## What I do
|
||||||
|
|
||||||
|
- Operate strictly within the `orchestrator/` folder.
|
||||||
|
- Produce a Markdown report describing:
|
||||||
|
1. **Duplication / near-duplication**
|
||||||
|
2. **Over-complex functions / “god modules”**
|
||||||
|
3. **Inconsistent error handling and logging**
|
||||||
|
- Use `npm run check:all` as the authoritative signal for tests + linting.
|
||||||
|
- Do **not** change code, configuration, scripts, or dependencies. This skill outputs a report only.
|
||||||
|
|
||||||
|
## When to use me
|
||||||
|
|
||||||
|
Use this when you want an evidence-driven code quality assessment and a prioritized plan for remediation **without implementing changes yet**.
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Work only in `orchestrator/`. Do not analyze or reference other folders except when required to understand imports into `orchestrator/`.
|
||||||
|
- Do not propose changes to tooling. Assume `npm run check:all` is the existing gate.
|
||||||
|
- Do not perform refactors. Do not open PRs. Do not rewrite files. Report findings and recommended fixes only.
|
||||||
|
|
||||||
|
## Inputs I expect
|
||||||
|
|
||||||
|
- Repository access with an `orchestrator/` directory.
|
||||||
|
- Ability to run `npm run check:all` from within `orchestrator/`.
|
||||||
|
- (Optional but recommended) Output logs from `npm run check:all` if the environment cannot run commands.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
1. **Set working area**
|
||||||
|
- Ensure all investigation is scoped to `orchestrator/`.
|
||||||
|
|
||||||
|
2. **Execute quality gate**
|
||||||
|
- Run `npm run check:all` in `orchestrator/`.
|
||||||
|
- Capture:
|
||||||
|
- lint errors/warnings (rule IDs, counts, file paths)
|
||||||
|
- failing tests (test names, stack traces, determinism vs flakiness indicators)
|
||||||
|
- coverage output (if produced by the command)
|
||||||
|
|
||||||
|
3. **Assess duplication**
|
||||||
|
- Identify copy/paste and near-duplicate logic patterns in `orchestrator/`:
|
||||||
|
- repeated parsing/validation
|
||||||
|
- repeated error mapping and response shaping
|
||||||
|
- repeated API call wrappers / fetch patterns
|
||||||
|
- repeated UI hooks/components with minor differences (if applicable)
|
||||||
|
- Prefer evidence:
|
||||||
|
- “same structure appears in files A/B/C”
|
||||||
|
- “multiple implementations differ only by X/Y flags”
|
||||||
|
- If the repo already includes duplication tooling, use it; otherwise rely on code inspection and minimal search.
|
||||||
|
|
||||||
|
4. **Assess complexity hotspots**
|
||||||
|
- Identify functions/modules with:
|
||||||
|
- deep nesting
|
||||||
|
- long functions
|
||||||
|
- mixed responsibilities (validation + IO + transformation + logging)
|
||||||
|
- difficult-to-test implicit dependencies
|
||||||
|
- Provide concrete hotspots (file paths, function names) and why they’re hotspots.
|
||||||
|
|
||||||
|
5. **Assess error handling and logging consistency**
|
||||||
|
- Check for:
|
||||||
|
- multiple error shapes (different status codes/messages for same category)
|
||||||
|
- swallowed errors vs raw throws
|
||||||
|
- missing context/correlation fields in logs
|
||||||
|
- potentially sensitive data in logs
|
||||||
|
- Identify boundary points (handlers, adapters) where standardization should occur.
|
||||||
|
|
||||||
|
6. **Synthesize a fix plan**
|
||||||
|
- Provide a prioritized plan with blast radius assessment and test needs.
|
||||||
|
- Do not implement. Do not request changes. Report recommended actions only.
|
||||||
|
|
||||||
|
## Report output format
|
||||||
|
|
||||||
|
Create a single Markdown report containing these sections:
|
||||||
|
|
||||||
|
### 1. Scope and constraints
|
||||||
|
|
||||||
|
- Confirm `orchestrator/` scope and reliance on `npm run check:all`.
|
||||||
|
|
||||||
|
### 2. Evidence summary
|
||||||
|
|
||||||
|
- `npm run check:all` outcome:
|
||||||
|
- Lint: top rule IDs + counts + representative file paths
|
||||||
|
- Tests: failing tests list + deterministic vs flaky notes
|
||||||
|
- Coverage: summary if present
|
||||||
|
|
||||||
|
### 3. What’s wrong and why it matters
|
||||||
|
|
||||||
|
Include these subsections (at minimum):
|
||||||
|
|
||||||
|
#### 3.1 Duplication and near-duplication
|
||||||
|
|
||||||
|
- Symptoms observed (with file-level examples)
|
||||||
|
- Why it matters in this repo
|
||||||
|
- What should be done to fix it (safe refactor tactics + tests needed)
|
||||||
|
|
||||||
|
#### 3.2 Over-complex functions and “god modules”
|
||||||
|
|
||||||
|
- Hotspots (file paths + functions)
|
||||||
|
- Risk explanation (bug surface, change friction)
|
||||||
|
- What should be done (decomposition targets + boundaries + tests)
|
||||||
|
|
||||||
|
#### 3.3 Inconsistent error handling and logging
|
||||||
|
|
||||||
|
- Inconsistencies found (examples)
|
||||||
|
- Operational/security impact
|
||||||
|
- What should be done (standard error types/mapping + logging conventions)
|
||||||
|
|
||||||
|
### 4. Fix strategy (prioritized)
|
||||||
|
|
||||||
|
- Phase 1 — Reduce duplication safely
|
||||||
|
- Phase 2 — Decrease complexity in hotspots
|
||||||
|
- Phase 3 — Standardize error handling and logging
|
||||||
|
|
||||||
|
Each phase must include:
|
||||||
|
|
||||||
|
- Ordered list of actions
|
||||||
|
- Expected blast radius (low/medium/high)
|
||||||
|
- Required test additions (if any)
|
||||||
|
- Validation steps (how to confirm improvement via `npm run check:all`)
|
||||||
|
|
||||||
|
### 5. Definition of done
|
||||||
|
|
||||||
|
- `npm run check:all` passes reliably
|
||||||
|
- Measurable reduction in targeted duplication clusters
|
||||||
|
- Complexity hotspots reduced or isolated behind clear boundaries
|
||||||
|
- Error/logging behavior consistent at boundaries
|
||||||
|
- No coverage regressions (if coverage is tracked)
|
||||||
|
|
||||||
|
## Quality bar for recommendations
|
||||||
|
|
||||||
|
- Be specific: include file paths, function names, rule IDs, and observed patterns.
|
||||||
|
- Be conservative: prefer small, behavior-preserving refactors in the plan.
|
||||||
|
- Avoid “make it green” shortcuts:
|
||||||
|
- do not recommend disabling lint rules/tests as a first move
|
||||||
|
- if quarantining a flaky test is suggested, include a concrete follow-up requirement and deadline policy
|
||||||
@ -8,6 +8,7 @@ import type {
|
|||||||
ApplicationStage,
|
ApplicationStage,
|
||||||
ApplicationTask,
|
ApplicationTask,
|
||||||
AppSettings,
|
AppSettings,
|
||||||
|
BackupInfo,
|
||||||
CreateJobInput,
|
CreateJobInput,
|
||||||
Job,
|
Job,
|
||||||
JobOutcome,
|
JobOutcome,
|
||||||
@ -474,3 +475,25 @@ export async function updateVisaSponsorList(): Promise<{
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Bulk operations (intentionally none - processing is manual)
|
// Bulk operations (intentionally none - processing is manual)
|
||||||
|
|
||||||
|
// Backup API
|
||||||
|
export interface BackupListResponse {
|
||||||
|
backups: BackupInfo[];
|
||||||
|
nextScheduled: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function getBackups(): Promise<BackupListResponse> {
|
||||||
|
return fetchApi<BackupListResponse>("/backups");
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function createManualBackup(): Promise<BackupInfo> {
|
||||||
|
return fetchApi<BackupInfo>("/backups", {
|
||||||
|
method: "POST",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function deleteBackup(filename: string): Promise<void> {
|
||||||
|
await fetchApi<void>(`/backups/${encodeURIComponent(filename)}`, {
|
||||||
|
method: "DELETE",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|||||||
@ -114,6 +114,16 @@ const baseSettings: AppSettings = {
|
|||||||
webhookSecretHint: null,
|
webhookSecretHint: null,
|
||||||
basicAuthActive: false,
|
basicAuthActive: false,
|
||||||
rxresumeBaseResumeId: null,
|
rxresumeBaseResumeId: null,
|
||||||
|
// Backup settings
|
||||||
|
backupEnabled: false,
|
||||||
|
defaultBackupEnabled: false,
|
||||||
|
overrideBackupEnabled: null,
|
||||||
|
backupHour: 2,
|
||||||
|
defaultBackupHour: 2,
|
||||||
|
overrideBackupHour: null,
|
||||||
|
backupMaxCount: 5,
|
||||||
|
defaultBackupMaxCount: 5,
|
||||||
|
overrideBackupMaxCount: null,
|
||||||
};
|
};
|
||||||
|
|
||||||
const renderPage = () => {
|
const renderPage = () => {
|
||||||
|
|||||||
@ -1,5 +1,6 @@
|
|||||||
import * as api from "@client/api";
|
import * as api from "@client/api";
|
||||||
import { PageHeader } from "@client/components/layout";
|
import { PageHeader } from "@client/components/layout";
|
||||||
|
import { BackupSettingsSection } from "@client/pages/settings/components/BackupSettingsSection";
|
||||||
import { DangerZoneSection } from "@client/pages/settings/components/DangerZoneSection";
|
import { DangerZoneSection } from "@client/pages/settings/components/DangerZoneSection";
|
||||||
import { DisplaySettingsSection } from "@client/pages/settings/components/DisplaySettingsSection";
|
import { DisplaySettingsSection } from "@client/pages/settings/components/DisplaySettingsSection";
|
||||||
import { EnvironmentSettingsSection } from "@client/pages/settings/components/EnvironmentSettingsSection";
|
import { EnvironmentSettingsSection } from "@client/pages/settings/components/EnvironmentSettingsSection";
|
||||||
@ -22,13 +23,14 @@ import {
|
|||||||
} from "@shared/settings-schema";
|
} from "@shared/settings-schema";
|
||||||
import type {
|
import type {
|
||||||
AppSettings,
|
AppSettings,
|
||||||
|
BackupInfo,
|
||||||
JobStatus,
|
JobStatus,
|
||||||
ResumeProjectCatalogItem,
|
ResumeProjectCatalogItem,
|
||||||
ResumeProjectsSettings,
|
ResumeProjectsSettings,
|
||||||
} from "@shared/types";
|
} from "@shared/types";
|
||||||
import { Settings } from "lucide-react";
|
import { Settings } from "lucide-react";
|
||||||
import type React from "react";
|
import type React from "react";
|
||||||
import { useEffect, useState } from "react";
|
import { useCallback, useEffect, useState } from "react";
|
||||||
import { FormProvider, type Resolver, useForm } from "react-hook-form";
|
import { FormProvider, type Resolver, useForm } from "react-hook-form";
|
||||||
import { toast } from "sonner";
|
import { toast } from "sonner";
|
||||||
import { Accordion } from "@/components/ui/accordion";
|
import { Accordion } from "@/components/ui/accordion";
|
||||||
@ -67,6 +69,9 @@ const DEFAULT_FORM_VALUES: UpdateSettingsInput = {
|
|||||||
ukvisajobsPassword: "",
|
ukvisajobsPassword: "",
|
||||||
webhookSecret: "",
|
webhookSecret: "",
|
||||||
enableBasicAuth: false,
|
enableBasicAuth: false,
|
||||||
|
backupEnabled: null,
|
||||||
|
backupHour: null,
|
||||||
|
backupMaxCount: null,
|
||||||
};
|
};
|
||||||
|
|
||||||
type LlmProviderValue = LlmProviderId | null;
|
type LlmProviderValue = LlmProviderId | null;
|
||||||
@ -107,6 +112,9 @@ const NULL_SETTINGS_PAYLOAD: UpdateSettingsInput = {
|
|||||||
ukvisajobsPassword: null,
|
ukvisajobsPassword: null,
|
||||||
webhookSecret: null,
|
webhookSecret: null,
|
||||||
enableBasicAuth: undefined,
|
enableBasicAuth: undefined,
|
||||||
|
backupEnabled: null,
|
||||||
|
backupHour: null,
|
||||||
|
backupMaxCount: null,
|
||||||
};
|
};
|
||||||
|
|
||||||
const mapSettingsToForm = (data: AppSettings): UpdateSettingsInput => ({
|
const mapSettingsToForm = (data: AppSettings): UpdateSettingsInput => ({
|
||||||
@ -141,6 +149,9 @@ const mapSettingsToForm = (data: AppSettings): UpdateSettingsInput => ({
|
|||||||
ukvisajobsPassword: "",
|
ukvisajobsPassword: "",
|
||||||
webhookSecret: "",
|
webhookSecret: "",
|
||||||
enableBasicAuth: data.basicAuthActive,
|
enableBasicAuth: data.basicAuthActive,
|
||||||
|
backupEnabled: data.overrideBackupEnabled,
|
||||||
|
backupHour: data.overrideBackupHour,
|
||||||
|
backupMaxCount: data.overrideBackupMaxCount,
|
||||||
});
|
});
|
||||||
|
|
||||||
const normalizeString = (value: string | null | undefined) => {
|
const normalizeString = (value: string | null | undefined) => {
|
||||||
@ -307,6 +318,21 @@ const getDerivedSettings = (settings: AppSettings | null) => {
|
|||||||
|
|
||||||
profileProjects,
|
profileProjects,
|
||||||
maxProjectsTotal: profileProjects.length,
|
maxProjectsTotal: profileProjects.length,
|
||||||
|
|
||||||
|
backup: {
|
||||||
|
backupEnabled: {
|
||||||
|
effective: settings?.backupEnabled ?? false,
|
||||||
|
default: settings?.defaultBackupEnabled ?? false,
|
||||||
|
},
|
||||||
|
backupHour: {
|
||||||
|
effective: settings?.backupHour ?? 2,
|
||||||
|
default: settings?.defaultBackupHour ?? 2,
|
||||||
|
},
|
||||||
|
backupMaxCount: {
|
||||||
|
effective: settings?.backupMaxCount ?? 5,
|
||||||
|
default: settings?.defaultBackupMaxCount ?? 5,
|
||||||
|
},
|
||||||
|
},
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
@ -326,6 +352,13 @@ export const SettingsPage: React.FC = () => {
|
|||||||
const [isFetchingRxResumeProjects, setIsFetchingRxResumeProjects] =
|
const [isFetchingRxResumeProjects, setIsFetchingRxResumeProjects] =
|
||||||
useState(false);
|
useState(false);
|
||||||
|
|
||||||
|
// Backup state
|
||||||
|
const [backups, setBackups] = useState<BackupInfo[]>([]);
|
||||||
|
const [nextScheduled, setNextScheduled] = useState<string | null>(null);
|
||||||
|
const [isLoadingBackups, setIsLoadingBackups] = useState(false);
|
||||||
|
const [isCreatingBackup, setIsCreatingBackup] = useState(false);
|
||||||
|
const [isDeletingBackup, setIsDeletingBackup] = useState(false);
|
||||||
|
|
||||||
const methods = useForm<UpdateSettingsInput>({
|
const methods = useForm<UpdateSettingsInput>({
|
||||||
resolver: zodResolver(
|
resolver: zodResolver(
|
||||||
updateSettingsSchema,
|
updateSettingsSchema,
|
||||||
@ -446,8 +479,68 @@ export const SettingsPage: React.FC = () => {
|
|||||||
envSettings,
|
envSettings,
|
||||||
defaultResumeProjects,
|
defaultResumeProjects,
|
||||||
profileProjects,
|
profileProjects,
|
||||||
|
backup,
|
||||||
} = derived;
|
} = derived;
|
||||||
|
|
||||||
|
// Backup functions
|
||||||
|
const loadBackups = useCallback(async () => {
|
||||||
|
setIsLoadingBackups(true);
|
||||||
|
try {
|
||||||
|
const response = await api.getBackups();
|
||||||
|
setBackups(response.backups);
|
||||||
|
setNextScheduled(response.nextScheduled);
|
||||||
|
} catch (error) {
|
||||||
|
const message =
|
||||||
|
error instanceof Error ? error.message : "Failed to load backups";
|
||||||
|
toast.error(message);
|
||||||
|
} finally {
|
||||||
|
setIsLoadingBackups(false);
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const handleCreateBackup = async () => {
|
||||||
|
setIsCreatingBackup(true);
|
||||||
|
try {
|
||||||
|
await api.createManualBackup();
|
||||||
|
toast.success("Backup created successfully");
|
||||||
|
await loadBackups();
|
||||||
|
} catch (error) {
|
||||||
|
const message =
|
||||||
|
error instanceof Error ? error.message : "Failed to create backup";
|
||||||
|
toast.error(message);
|
||||||
|
} finally {
|
||||||
|
setIsCreatingBackup(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleDeleteBackup = async (filename: string) => {
|
||||||
|
const confirmed = window.confirm(
|
||||||
|
`Delete backup "${filename}"? This action cannot be undone.`,
|
||||||
|
);
|
||||||
|
if (!confirmed) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
setIsDeletingBackup(true);
|
||||||
|
try {
|
||||||
|
await api.deleteBackup(filename);
|
||||||
|
toast.success("Backup deleted successfully");
|
||||||
|
await loadBackups();
|
||||||
|
} catch (error) {
|
||||||
|
const message =
|
||||||
|
error instanceof Error ? error.message : "Failed to delete backup";
|
||||||
|
toast.error(message);
|
||||||
|
} finally {
|
||||||
|
setIsDeletingBackup(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Load backups when settings are loaded
|
||||||
|
useEffect(() => {
|
||||||
|
if (settings) {
|
||||||
|
loadBackups();
|
||||||
|
}
|
||||||
|
}, [settings, loadBackups]);
|
||||||
|
|
||||||
const effectiveProfileProjects = rxResumeProjectsOverride ?? profileProjects;
|
const effectiveProfileProjects = rxResumeProjectsOverride ?? profileProjects;
|
||||||
const effectiveMaxProjectsTotal = effectiveProfileProjects.length;
|
const effectiveMaxProjectsTotal = effectiveProfileProjects.length;
|
||||||
|
|
||||||
@ -589,6 +682,15 @@ export const SettingsPage: React.FC = () => {
|
|||||||
jobspy.isRemote.default,
|
jobspy.isRemote.default,
|
||||||
),
|
),
|
||||||
showSponsorInfo: nullIfSame(data.showSponsorInfo, display.default),
|
showSponsorInfo: nullIfSame(data.showSponsorInfo, display.default),
|
||||||
|
backupEnabled: nullIfSame(
|
||||||
|
data.backupEnabled,
|
||||||
|
backup.backupEnabled.default,
|
||||||
|
),
|
||||||
|
backupHour: nullIfSame(data.backupHour, backup.backupHour.default),
|
||||||
|
backupMaxCount: nullIfSame(
|
||||||
|
data.backupMaxCount,
|
||||||
|
backup.backupMaxCount.default,
|
||||||
|
),
|
||||||
...envPayload,
|
...envPayload,
|
||||||
};
|
};
|
||||||
|
|
||||||
@ -751,6 +853,17 @@ export const SettingsPage: React.FC = () => {
|
|||||||
isLoading={isLoading}
|
isLoading={isLoading}
|
||||||
isSaving={isSaving}
|
isSaving={isSaving}
|
||||||
/>
|
/>
|
||||||
|
<BackupSettingsSection
|
||||||
|
values={backup}
|
||||||
|
backups={backups}
|
||||||
|
nextScheduled={nextScheduled}
|
||||||
|
isLoading={isLoading || isLoadingBackups}
|
||||||
|
isSaving={isSaving}
|
||||||
|
onCreateBackup={handleCreateBackup}
|
||||||
|
onDeleteBackup={handleDeleteBackup}
|
||||||
|
isCreatingBackup={isCreatingBackup}
|
||||||
|
isDeletingBackup={isDeletingBackup}
|
||||||
|
/>
|
||||||
<DangerZoneSection
|
<DangerZoneSection
|
||||||
statusesToClear={statusesToClear}
|
statusesToClear={statusesToClear}
|
||||||
toggleStatusToClear={toggleStatusToClear}
|
toggleStatusToClear={toggleStatusToClear}
|
||||||
|
|||||||
@ -0,0 +1,283 @@
|
|||||||
|
import { EmptyState, ListItem, ListPanel } from "@client/components/layout";
|
||||||
|
import { SettingsInput } from "@client/pages/settings/components/SettingsInput";
|
||||||
|
import type { BackupValues } from "@client/pages/settings/types";
|
||||||
|
import type { UpdateSettingsInput } from "@shared/settings-schema";
|
||||||
|
import type { BackupInfo } from "@shared/types";
|
||||||
|
import { Archive, Clock, Trash2 } from "lucide-react";
|
||||||
|
import type React from "react";
|
||||||
|
import { Controller, useFormContext } from "react-hook-form";
|
||||||
|
import {
|
||||||
|
AccordionContent,
|
||||||
|
AccordionItem,
|
||||||
|
AccordionTrigger,
|
||||||
|
} from "@/components/ui/accordion";
|
||||||
|
import { Badge } from "@/components/ui/badge";
|
||||||
|
import { Button } from "@/components/ui/button";
|
||||||
|
import { Checkbox } from "@/components/ui/checkbox";
|
||||||
|
import { Separator } from "@/components/ui/separator";
|
||||||
|
|
||||||
|
type BackupSettingsSectionProps = {
|
||||||
|
values: BackupValues;
|
||||||
|
backups: BackupInfo[];
|
||||||
|
nextScheduled: string | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
isSaving: boolean;
|
||||||
|
onCreateBackup: () => void;
|
||||||
|
onDeleteBackup: (filename: string) => void;
|
||||||
|
isCreatingBackup: boolean;
|
||||||
|
isDeletingBackup: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
const formatFileSize = (bytes: number): string => {
|
||||||
|
if (bytes < 1024) return `${bytes} B`;
|
||||||
|
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||||
|
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
|
||||||
|
};
|
||||||
|
|
||||||
|
const formatBackupDate = (dateString: string): string => {
|
||||||
|
const date = new Date(dateString);
|
||||||
|
return date.toLocaleString("en-US", {
|
||||||
|
timeZone: "UTC",
|
||||||
|
month: "short",
|
||||||
|
day: "numeric",
|
||||||
|
year: "numeric",
|
||||||
|
hour: "numeric",
|
||||||
|
minute: "2-digit",
|
||||||
|
hour12: true,
|
||||||
|
timeZoneName: "short",
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
export const BackupSettingsSection: React.FC<BackupSettingsSectionProps> = ({
|
||||||
|
values,
|
||||||
|
backups,
|
||||||
|
nextScheduled,
|
||||||
|
isLoading,
|
||||||
|
isSaving,
|
||||||
|
onCreateBackup,
|
||||||
|
onDeleteBackup,
|
||||||
|
isCreatingBackup,
|
||||||
|
isDeletingBackup,
|
||||||
|
}) => {
|
||||||
|
const { backupEnabled, backupHour, backupMaxCount } = values;
|
||||||
|
const { control, watch } = useFormContext<UpdateSettingsInput>();
|
||||||
|
|
||||||
|
// Watch the current form value to conditionally show/hide fields
|
||||||
|
const currentBackupEnabled = watch("backupEnabled") ?? backupEnabled.default;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<AccordionItem value="backup" className="border rounded-lg px-4">
|
||||||
|
<AccordionTrigger className="hover:no-underline py-4">
|
||||||
|
<span className="text-base font-semibold">Backup</span>
|
||||||
|
</AccordionTrigger>
|
||||||
|
<AccordionContent className="pb-4">
|
||||||
|
<div className="space-y-6">
|
||||||
|
{/* Enable automatic backups toggle */}
|
||||||
|
<div className="flex items-start space-x-3">
|
||||||
|
<Controller
|
||||||
|
name="backupEnabled"
|
||||||
|
control={control}
|
||||||
|
render={({ field }) => (
|
||||||
|
<Checkbox
|
||||||
|
id="backupEnabled"
|
||||||
|
checked={field.value ?? backupEnabled.default}
|
||||||
|
onCheckedChange={(checked) => {
|
||||||
|
field.onChange(
|
||||||
|
checked === "indeterminate" ? null : checked === true,
|
||||||
|
);
|
||||||
|
}}
|
||||||
|
disabled={isLoading || isSaving}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
<div className="flex flex-col gap-1.5">
|
||||||
|
<label
|
||||||
|
htmlFor="backupEnabled"
|
||||||
|
className="text-sm font-medium leading-none cursor-pointer"
|
||||||
|
>
|
||||||
|
Enable automatic backups
|
||||||
|
</label>
|
||||||
|
<p className="text-xs text-muted-foreground">
|
||||||
|
Automatically create database backups on a daily schedule.
|
||||||
|
Manual backups can always be created regardless of this setting.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{/* Backup settings - only shown when enabled */}
|
||||||
|
{currentBackupEnabled && (
|
||||||
|
<div className="grid gap-6 md:grid-cols-2 pl-7">
|
||||||
|
<Controller
|
||||||
|
name="backupHour"
|
||||||
|
control={control}
|
||||||
|
render={({ field }) => (
|
||||||
|
<SettingsInput
|
||||||
|
label="Backup Hour"
|
||||||
|
type="number"
|
||||||
|
inputProps={{
|
||||||
|
...field,
|
||||||
|
inputMode: "numeric",
|
||||||
|
min: 0,
|
||||||
|
max: 23,
|
||||||
|
value: field.value ?? backupHour.default,
|
||||||
|
onChange: (event) => {
|
||||||
|
const value = parseInt(event.target.value, 10);
|
||||||
|
if (Number.isNaN(value)) {
|
||||||
|
field.onChange(null);
|
||||||
|
} else {
|
||||||
|
field.onChange(Math.min(23, Math.max(0, value)));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
disabled={isLoading || isSaving}
|
||||||
|
helper={`Hour of the day (0-23) in UTC when automatic backups should run. Default: ${backupHour.default}:00 UTC.`}
|
||||||
|
current={`Effective: ${backupHour.effective}:00 UTC | Default: ${backupHour.default}:00 UTC`}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
|
||||||
|
<Controller
|
||||||
|
name="backupMaxCount"
|
||||||
|
control={control}
|
||||||
|
render={({ field }) => (
|
||||||
|
<SettingsInput
|
||||||
|
label="Max Backups to Keep"
|
||||||
|
type="number"
|
||||||
|
inputProps={{
|
||||||
|
...field,
|
||||||
|
inputMode: "numeric",
|
||||||
|
min: 1,
|
||||||
|
max: 5,
|
||||||
|
value: field.value ?? backupMaxCount.default,
|
||||||
|
onChange: (event) => {
|
||||||
|
const value = parseInt(event.target.value, 10);
|
||||||
|
if (Number.isNaN(value)) {
|
||||||
|
field.onChange(null);
|
||||||
|
} else {
|
||||||
|
field.onChange(Math.min(5, Math.max(1, value)));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}}
|
||||||
|
disabled={isLoading || isSaving}
|
||||||
|
helper={`Maximum number of automatic backups to retain (1-5). Older backups are deleted automatically. Default: ${backupMaxCount.default}.`}
|
||||||
|
current={`Effective: ${backupMaxCount.effective} | Default: ${backupMaxCount.default}`}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{/* Next scheduled backup */}
|
||||||
|
{currentBackupEnabled && nextScheduled && (
|
||||||
|
<div className="flex items-center gap-2 text-sm text-muted-foreground pl-7">
|
||||||
|
<Clock className="h-4 w-4" />
|
||||||
|
<span>
|
||||||
|
Next scheduled backup: {formatBackupDate(nextScheduled)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<Separator />
|
||||||
|
|
||||||
|
{/* Backup list */}
|
||||||
|
<div className="space-y-3">
|
||||||
|
<div className="flex items-center justify-between">
|
||||||
|
<div className="text-sm font-medium">Backup History</div>
|
||||||
|
<Button
|
||||||
|
size="sm"
|
||||||
|
onClick={onCreateBackup}
|
||||||
|
disabled={isLoading || isCreatingBackup || isDeletingBackup}
|
||||||
|
>
|
||||||
|
{isCreatingBackup ? "Creating..." : "Create Backup Now"}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<ListPanel
|
||||||
|
header={
|
||||||
|
<div className="flex items-center justify-between text-sm">
|
||||||
|
<span>
|
||||||
|
{backups.length} backup{backups.length !== 1 ? "s" : ""}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
>
|
||||||
|
{backups.length === 0 ? (
|
||||||
|
<EmptyState
|
||||||
|
icon={Archive}
|
||||||
|
title="No backups yet"
|
||||||
|
description="Create your first backup to protect your data."
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
backups.map((backup) => (
|
||||||
|
<ListItem
|
||||||
|
key={backup.filename}
|
||||||
|
className="flex items-center justify-between"
|
||||||
|
>
|
||||||
|
<div className="flex items-center gap-3 min-w-0">
|
||||||
|
<Archive className="h-4 w-4 text-muted-foreground shrink-0" />
|
||||||
|
<div className="min-w-0">
|
||||||
|
<div className="text-sm font-medium truncate">
|
||||||
|
{backup.filename}
|
||||||
|
</div>
|
||||||
|
<div className="text-xs text-muted-foreground">
|
||||||
|
{formatBackupDate(backup.createdAt)} ·{" "}
|
||||||
|
{formatFileSize(backup.size)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="flex items-center gap-2 shrink-0">
|
||||||
|
<Badge
|
||||||
|
variant={
|
||||||
|
backup.type === "auto" ? "secondary" : "default"
|
||||||
|
}
|
||||||
|
className="text-xs"
|
||||||
|
>
|
||||||
|
{backup.type === "auto" ? "Auto" : "Manual"}
|
||||||
|
</Badge>
|
||||||
|
<Button
|
||||||
|
variant="ghost"
|
||||||
|
size="icon"
|
||||||
|
className="h-8 w-8 text-destructive hover:text-destructive"
|
||||||
|
onClick={() => onDeleteBackup(backup.filename)}
|
||||||
|
disabled={isDeletingBackup || isCreatingBackup}
|
||||||
|
>
|
||||||
|
<Trash2 className="h-4 w-4" />
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</ListItem>
|
||||||
|
))
|
||||||
|
)}
|
||||||
|
</ListPanel>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Separator />
|
||||||
|
|
||||||
|
{/* Effective/Default values display */}
|
||||||
|
<div className="grid gap-2 text-sm sm:grid-cols-3">
|
||||||
|
<div>
|
||||||
|
<div className="text-xs text-muted-foreground">Enabled</div>
|
||||||
|
<div className="break-words font-mono text-xs">
|
||||||
|
Effective: {backupEnabled.effective ? "Yes" : "No"} | Default:{" "}
|
||||||
|
{backupEnabled.default ? "Yes" : "No"}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div className="text-xs text-muted-foreground">Hour</div>
|
||||||
|
<div className="break-words font-mono text-xs">
|
||||||
|
Effective: {backupHour.effective}:00 UTC | Default:{" "}
|
||||||
|
{backupHour.default}:00 UTC
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div className="text-xs text-muted-foreground">Max Count</div>
|
||||||
|
<div className="break-words font-mono text-xs">
|
||||||
|
Effective: {backupMaxCount.effective} | Default:{" "}
|
||||||
|
{backupMaxCount.default}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</AccordionContent>
|
||||||
|
</AccordionItem>
|
||||||
|
);
|
||||||
|
};
|
||||||
@ -42,3 +42,9 @@ export type EnvSettingsValues = {
|
|||||||
};
|
};
|
||||||
basicAuthActive: boolean;
|
basicAuthActive: boolean;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export type BackupValues = {
|
||||||
|
backupEnabled: EffectiveDefault<boolean>;
|
||||||
|
backupHour: EffectiveDefault<number>;
|
||||||
|
backupMaxCount: EffectiveDefault<number>;
|
||||||
|
};
|
||||||
|
|||||||
@ -3,6 +3,7 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { Router } from "express";
|
import { Router } from "express";
|
||||||
|
import { backupRouter } from "./routes/backup.js";
|
||||||
import { databaseRouter } from "./routes/database.js";
|
import { databaseRouter } from "./routes/database.js";
|
||||||
import { jobsRouter } from "./routes/jobs.js";
|
import { jobsRouter } from "./routes/jobs.js";
|
||||||
import { manualJobsRouter } from "./routes/manual-jobs.js";
|
import { manualJobsRouter } from "./routes/manual-jobs.js";
|
||||||
@ -26,3 +27,4 @@ apiRouter.use("/profile", profileRouter);
|
|||||||
apiRouter.use("/database", databaseRouter);
|
apiRouter.use("/database", databaseRouter);
|
||||||
apiRouter.use("/visa-sponsors", visaSponsorsRouter);
|
apiRouter.use("/visa-sponsors", visaSponsorsRouter);
|
||||||
apiRouter.use("/onboarding", onboardingRouter);
|
apiRouter.use("/onboarding", onboardingRouter);
|
||||||
|
apiRouter.use("/backups", backupRouter);
|
||||||
|
|||||||
122
orchestrator/src/server/api/routes/backup.test.ts
Normal file
122
orchestrator/src/server/api/routes/backup.test.ts
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
import fs from "node:fs";
|
||||||
|
import type { Server } from "node:http";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||||
|
import { startServer, stopServer } from "./test-utils.js";
|
||||||
|
|
||||||
|
describe.sequential("Backup API routes", () => {
|
||||||
|
let server: Server;
|
||||||
|
let baseUrl: string;
|
||||||
|
let closeDb: () => void;
|
||||||
|
let tempDir: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
({ server, baseUrl, closeDb, tempDir } = await startServer());
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await stopServer({ server, closeDb, tempDir });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("GET /api/backups", () => {
|
||||||
|
it("should return empty array when no backups exist", async () => {
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups`);
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
expect(body.success).toBe(true);
|
||||||
|
expect(body.data.backups).toEqual([]);
|
||||||
|
expect(body.data.nextScheduled).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should list backups with metadata", async () => {
|
||||||
|
// Create a backup first
|
||||||
|
await fetch(`${baseUrl}/api/backups`, { method: "POST" });
|
||||||
|
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups`);
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
expect(body.success).toBe(true);
|
||||||
|
expect(body.data.backups).toHaveLength(1);
|
||||||
|
expect(body.data.backups[0]).toHaveProperty("filename");
|
||||||
|
expect(body.data.backups[0]).toHaveProperty("type", "manual");
|
||||||
|
expect(body.data.backups[0]).toHaveProperty("size");
|
||||||
|
expect(body.data.backups[0]).toHaveProperty("createdAt");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("POST /api/backups", () => {
|
||||||
|
it("should create a manual backup", async () => {
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups`, { method: "POST" });
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
expect(body.success).toBe(true);
|
||||||
|
expect(body.data.type).toBe("manual");
|
||||||
|
expect(body.data.filename).toMatch(
|
||||||
|
/^jobs_manual_\d{4}_\d{2}_\d{2}_\d{2}_\d{2}_\d{2}\.db$/,
|
||||||
|
);
|
||||||
|
expect(body.data.size).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should return error if database does not exist", async () => {
|
||||||
|
// Delete the database
|
||||||
|
await fs.promises.unlink(`${tempDir}/jobs.db`);
|
||||||
|
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups`, { method: "POST" });
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(500);
|
||||||
|
expect(body.success).toBe(false);
|
||||||
|
expect(body.error).toContain("Database file not found");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("DELETE /api/backups/:filename", () => {
|
||||||
|
it("should delete a backup", async () => {
|
||||||
|
// Create a backup first
|
||||||
|
const createRes = await fetch(`${baseUrl}/api/backups`, {
|
||||||
|
method: "POST",
|
||||||
|
});
|
||||||
|
const createBody = await createRes.json();
|
||||||
|
const filename = createBody.data.filename;
|
||||||
|
|
||||||
|
// Delete the backup
|
||||||
|
const deleteRes = await fetch(`${baseUrl}/api/backups/${filename}`, {
|
||||||
|
method: "DELETE",
|
||||||
|
});
|
||||||
|
const deleteBody = await deleteRes.json();
|
||||||
|
|
||||||
|
expect(deleteRes.status).toBe(200);
|
||||||
|
expect(deleteBody.success).toBe(true);
|
||||||
|
expect(deleteBody.message).toContain("deleted successfully");
|
||||||
|
|
||||||
|
// Verify it's gone
|
||||||
|
const listRes = await fetch(`${baseUrl}/api/backups`);
|
||||||
|
const listBody = await listRes.json();
|
||||||
|
expect(listBody.data.backups).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should return 404 for non-existent backup", async () => {
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups/jobs_2026_01_01.db`, {
|
||||||
|
method: "DELETE",
|
||||||
|
});
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(404);
|
||||||
|
expect(body.success).toBe(false);
|
||||||
|
expect(body.error).toContain("not found");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should return 400 for invalid filename", async () => {
|
||||||
|
const res = await fetch(`${baseUrl}/api/backups/invalid_filename.txt`, {
|
||||||
|
method: "DELETE",
|
||||||
|
});
|
||||||
|
const body = await res.json();
|
||||||
|
|
||||||
|
expect(res.status).toBe(400);
|
||||||
|
expect(body.success).toBe(false);
|
||||||
|
expect(body.error).toContain("Invalid");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
93
orchestrator/src/server/api/routes/backup.ts
Normal file
93
orchestrator/src/server/api/routes/backup.ts
Normal file
@ -0,0 +1,93 @@
|
|||||||
|
import {
|
||||||
|
createBackup,
|
||||||
|
deleteBackup,
|
||||||
|
getNextBackupTime,
|
||||||
|
listBackups,
|
||||||
|
} from "@server/services/backup/index.js";
|
||||||
|
import { type Request, type Response, Router } from "express";
|
||||||
|
|
||||||
|
export const backupRouter = Router();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GET /api/backups - List all backups with metadata
|
||||||
|
*/
|
||||||
|
backupRouter.get("/", async (_req: Request, res: Response) => {
|
||||||
|
try {
|
||||||
|
const backups = await listBackups();
|
||||||
|
const nextScheduled = getNextBackupTime();
|
||||||
|
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
data: {
|
||||||
|
backups,
|
||||||
|
nextScheduled,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : "Unknown error";
|
||||||
|
console.error("❌ [backup-api] Failed to list backups:", error);
|
||||||
|
res.status(500).json({ success: false, error: message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* POST /api/backups - Create a manual backup
|
||||||
|
*/
|
||||||
|
backupRouter.post("/", async (_req: Request, res: Response) => {
|
||||||
|
try {
|
||||||
|
const filename = await createBackup("manual");
|
||||||
|
const backups = await listBackups();
|
||||||
|
const backup = backups.find((b) => b.filename === filename);
|
||||||
|
|
||||||
|
if (!backup) {
|
||||||
|
throw new Error("Backup was created but not found in list");
|
||||||
|
}
|
||||||
|
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
data: backup,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : "Unknown error";
|
||||||
|
console.error("❌ [backup-api] Failed to create backup:", error);
|
||||||
|
res.status(500).json({ success: false, error: message });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* DELETE /api/backups/:filename - Delete a specific backup
|
||||||
|
*/
|
||||||
|
backupRouter.delete("/:filename", async (req: Request, res: Response) => {
|
||||||
|
try {
|
||||||
|
const { filename } = req.params;
|
||||||
|
|
||||||
|
if (!filename) {
|
||||||
|
res.status(400).json({
|
||||||
|
success: false,
|
||||||
|
error: "Filename is required",
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await deleteBackup(filename);
|
||||||
|
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
message: `Backup ${filename} deleted successfully`,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : "Unknown error";
|
||||||
|
console.error(
|
||||||
|
`❌ [backup-api] Failed to delete backup ${req.params.filename}:`,
|
||||||
|
error,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (message.includes("not found")) {
|
||||||
|
res.status(404).json({ success: false, error: message });
|
||||||
|
} else if (message.includes("Invalid")) {
|
||||||
|
res.status(400).json({ success: false, error: message });
|
||||||
|
} else {
|
||||||
|
res.status(500).json({ success: false, error: message });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
@ -1,4 +1,5 @@
|
|||||||
import * as settingsRepo from "@server/repositories/settings.js";
|
import * as settingsRepo from "@server/repositories/settings.js";
|
||||||
|
import { setBackupSettings } from "@server/services/backup/index.js";
|
||||||
import {
|
import {
|
||||||
applyEnvValue,
|
applyEnvValue,
|
||||||
normalizeEnvInput,
|
normalizeEnvInput,
|
||||||
@ -328,9 +329,53 @@ settingsRouter.patch("/", async (req: Request, res: Response) => {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Backup settings
|
||||||
|
if ("backupEnabled" in input) {
|
||||||
|
const val = input.backupEnabled ?? null;
|
||||||
|
promises.push(
|
||||||
|
settingsRepo.setSetting(
|
||||||
|
"backupEnabled",
|
||||||
|
val !== null ? (val ? "1" : "0") : null,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ("backupHour" in input) {
|
||||||
|
const val = input.backupHour ?? null;
|
||||||
|
promises.push(
|
||||||
|
settingsRepo.setSetting(
|
||||||
|
"backupHour",
|
||||||
|
val !== null ? String(val) : null,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ("backupMaxCount" in input) {
|
||||||
|
const val = input.backupMaxCount ?? null;
|
||||||
|
promises.push(
|
||||||
|
settingsRepo.setSetting(
|
||||||
|
"backupMaxCount",
|
||||||
|
val !== null ? String(val) : null,
|
||||||
|
),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
await Promise.all(promises);
|
await Promise.all(promises);
|
||||||
|
|
||||||
const data = await getEffectiveSettings();
|
const data = await getEffectiveSettings();
|
||||||
|
|
||||||
|
// Update backup scheduler if backup settings changed
|
||||||
|
if (
|
||||||
|
"backupEnabled" in input ||
|
||||||
|
"backupHour" in input ||
|
||||||
|
"backupMaxCount" in input
|
||||||
|
) {
|
||||||
|
setBackupSettings({
|
||||||
|
enabled: data.backupEnabled,
|
||||||
|
hour: data.backupHour,
|
||||||
|
maxCount: data.backupMaxCount,
|
||||||
|
});
|
||||||
|
}
|
||||||
res.json({ success: true, data });
|
res.json({ success: true, data });
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
const message = error instanceof Error ? error.message : "Unknown error";
|
const message = error instanceof Error ? error.message : "Unknown error";
|
||||||
|
|||||||
@ -4,6 +4,12 @@
|
|||||||
|
|
||||||
import "./config/env.js";
|
import "./config/env.js";
|
||||||
import { createApp } from "./app.js";
|
import { createApp } from "./app.js";
|
||||||
|
import * as settingsRepo from "./repositories/settings.js";
|
||||||
|
import {
|
||||||
|
getBackupSettings,
|
||||||
|
setBackupSettings,
|
||||||
|
startBackupScheduler,
|
||||||
|
} from "./services/backup/index.js";
|
||||||
import { applyStoredEnvOverrides } from "./services/envSettings.js";
|
import { applyStoredEnvOverrides } from "./services/envSettings.js";
|
||||||
import { initialize as initializeVisaSponsors } from "./services/visa-sponsors/index.js";
|
import { initialize as initializeVisaSponsors } from "./services/visa-sponsors/index.js";
|
||||||
|
|
||||||
@ -35,6 +41,45 @@ async function startServer() {
|
|||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.warn("⚠️ Failed to initialize visa sponsors service:", error);
|
console.warn("⚠️ Failed to initialize visa sponsors service:", error);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Initialize backup service (load settings and start scheduler if enabled)
|
||||||
|
try {
|
||||||
|
const backupEnabled = await settingsRepo.getSetting("backupEnabled");
|
||||||
|
const backupHour = await settingsRepo.getSetting("backupHour");
|
||||||
|
const backupMaxCount = await settingsRepo.getSetting("backupMaxCount");
|
||||||
|
|
||||||
|
const parsedHour = backupHour ? parseInt(backupHour, 10) : NaN;
|
||||||
|
const parsedMaxCount = backupMaxCount
|
||||||
|
? parseInt(backupMaxCount, 10)
|
||||||
|
: NaN;
|
||||||
|
const safeHour = Number.isNaN(parsedHour)
|
||||||
|
? 2
|
||||||
|
: Math.min(23, Math.max(0, parsedHour));
|
||||||
|
const safeMaxCount = Number.isNaN(parsedMaxCount)
|
||||||
|
? 5
|
||||||
|
: Math.min(5, Math.max(1, parsedMaxCount));
|
||||||
|
|
||||||
|
setBackupSettings({
|
||||||
|
enabled: backupEnabled === "true" || backupEnabled === "1",
|
||||||
|
hour: safeHour,
|
||||||
|
maxCount: safeMaxCount,
|
||||||
|
});
|
||||||
|
|
||||||
|
startBackupScheduler();
|
||||||
|
|
||||||
|
const settings = getBackupSettings();
|
||||||
|
if (settings.enabled) {
|
||||||
|
console.log(
|
||||||
|
`✅ Backup scheduler started (hour: ${settings.hour}, max: ${settings.maxCount})`,
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
console.log(
|
||||||
|
"ℹ️ Backups disabled. Enable in settings to schedule automatic backups.",
|
||||||
|
);
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
console.warn("⚠️ Failed to initialize backup service:", error);
|
||||||
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -37,7 +37,10 @@ export type SettingKey =
|
|||||||
| "basicAuthPassword"
|
| "basicAuthPassword"
|
||||||
| "ukvisajobsEmail"
|
| "ukvisajobsEmail"
|
||||||
| "ukvisajobsPassword"
|
| "ukvisajobsPassword"
|
||||||
| "webhookSecret";
|
| "webhookSecret"
|
||||||
|
| "backupEnabled"
|
||||||
|
| "backupHour"
|
||||||
|
| "backupMaxCount";
|
||||||
|
|
||||||
export async function getSetting(key: SettingKey): Promise<string | null> {
|
export async function getSetting(key: SettingKey): Promise<string | null> {
|
||||||
const [row] = await db.select().from(settings).where(eq(settings.key, key));
|
const [row] = await db.select().from(settings).where(eq(settings.key, key));
|
||||||
|
|||||||
366
orchestrator/src/server/services/backup/index.test.ts
Normal file
366
orchestrator/src/server/services/backup/index.test.ts
Normal file
@ -0,0 +1,366 @@
|
|||||||
|
import fs from "node:fs";
|
||||||
|
import os from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import Database from "better-sqlite3";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import * as backup from "./index.js";
|
||||||
|
|
||||||
|
// Mock the dataDir module
|
||||||
|
vi.mock("../../config/dataDir.js", () => ({
|
||||||
|
getDataDir: vi.fn(),
|
||||||
|
}));
|
||||||
|
|
||||||
|
import { getDataDir } from "../../config/dataDir.js";
|
||||||
|
|
||||||
|
describe("Backup Service", () => {
|
||||||
|
let tempDir: string;
|
||||||
|
let dbPath: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
// Create temp directory for tests
|
||||||
|
tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), "backup-test-"));
|
||||||
|
dbPath = path.join(tempDir, "jobs.db");
|
||||||
|
|
||||||
|
// Create a real SQLite database file for backup() to work.
|
||||||
|
const db = new Database(dbPath);
|
||||||
|
try {
|
||||||
|
db.exec(
|
||||||
|
[
|
||||||
|
"PRAGMA journal_mode = DELETE;",
|
||||||
|
"CREATE TABLE IF NOT EXISTS test_items (id INTEGER PRIMARY KEY, name TEXT NOT NULL);",
|
||||||
|
"DELETE FROM test_items;",
|
||||||
|
"INSERT INTO test_items (name) VALUES ('alpha');",
|
||||||
|
].join("\n"),
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
db.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Mock getDataDir to return temp directory
|
||||||
|
vi.mocked(getDataDir).mockReturnValue(tempDir);
|
||||||
|
|
||||||
|
// Reset backup settings
|
||||||
|
backup.setBackupSettings({ enabled: false, hour: 2, maxCount: 5 });
|
||||||
|
backup.stopBackupScheduler();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
// Clean up temp directory
|
||||||
|
try {
|
||||||
|
await fs.promises.rm(tempDir, { recursive: true, force: true });
|
||||||
|
} catch {
|
||||||
|
// Ignore cleanup errors
|
||||||
|
}
|
||||||
|
vi.clearAllMocks();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("createBackup", () => {
|
||||||
|
it("should create an automatic backup with correct filename format", async () => {
|
||||||
|
const filename = await backup.createBackup("auto");
|
||||||
|
|
||||||
|
// Check filename format: jobs_YYYY_MM_DD.db
|
||||||
|
expect(filename).toMatch(/^jobs_\d{4}_\d{2}_\d{2}\.db$/);
|
||||||
|
|
||||||
|
// Check file was created
|
||||||
|
const backupPath = path.join(tempDir, filename);
|
||||||
|
expect(fs.existsSync(backupPath)).toBe(true);
|
||||||
|
|
||||||
|
// Check backup is a valid SQLite database with expected data
|
||||||
|
const backupDb = new Database(backupPath, {
|
||||||
|
readonly: true,
|
||||||
|
fileMustExist: true,
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
const row = backupDb
|
||||||
|
.prepare("SELECT name FROM test_items ORDER BY id LIMIT 1")
|
||||||
|
.get() as { name: string } | undefined;
|
||||||
|
expect(row?.name).toBe("alpha");
|
||||||
|
} finally {
|
||||||
|
backupDb.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should create a manual backup with correct filename format", async () => {
|
||||||
|
const filename = await backup.createBackup("manual");
|
||||||
|
|
||||||
|
// Check filename format: jobs_manual_YYYY_MM_DD_HH_MM_SS.db
|
||||||
|
expect(filename).toMatch(
|
||||||
|
/^jobs_manual_\d{4}_\d{2}_\d{2}_\d{2}_\d{2}_\d{2}\.db$/,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Check file was created
|
||||||
|
const backupPath = path.join(tempDir, filename);
|
||||||
|
expect(fs.existsSync(backupPath)).toBe(true);
|
||||||
|
|
||||||
|
const backupDb = new Database(backupPath, {
|
||||||
|
readonly: true,
|
||||||
|
fileMustExist: true,
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
const count = backupDb
|
||||||
|
.prepare("SELECT COUNT(*) as count FROM test_items")
|
||||||
|
.get() as { count: number };
|
||||||
|
expect(count.count).toBe(1);
|
||||||
|
} finally {
|
||||||
|
backupDb.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should add a suffix when manual backup name collides", async () => {
|
||||||
|
// Only fake Date to keep async I/O (used by better-sqlite3 backup) real.
|
||||||
|
vi.useFakeTimers({ toFake: ["Date"] });
|
||||||
|
try {
|
||||||
|
vi.setSystemTime(new Date("2026-01-15T12:30:45Z"));
|
||||||
|
|
||||||
|
const first = await backup.createBackup("manual");
|
||||||
|
const second = await backup.createBackup("manual");
|
||||||
|
|
||||||
|
expect(first).toBe("jobs_manual_2026_01_15_12_30_45.db");
|
||||||
|
expect(second).toBe("jobs_manual_2026_01_15_12_30_45_1.db");
|
||||||
|
expect(fs.existsSync(path.join(tempDir, second))).toBe(true);
|
||||||
|
} finally {
|
||||||
|
vi.useRealTimers();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should throw error if database does not exist", async () => {
|
||||||
|
// Delete the database
|
||||||
|
await fs.promises.unlink(dbPath);
|
||||||
|
|
||||||
|
await expect(backup.createBackup("auto")).rejects.toThrow(
|
||||||
|
"Database file not found",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("listBackups", () => {
|
||||||
|
it("should return empty array when no backups exist", async () => {
|
||||||
|
const backups = await backup.listBackups();
|
||||||
|
expect(backups).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should list all backups with metadata", async () => {
|
||||||
|
// Create some backups
|
||||||
|
await backup.createBackup("auto");
|
||||||
|
await backup.createBackup("manual");
|
||||||
|
|
||||||
|
const backups = await backup.listBackups();
|
||||||
|
|
||||||
|
expect(backups).toHaveLength(2);
|
||||||
|
expect(backups[0]).toHaveProperty("filename");
|
||||||
|
expect(backups[0]).toHaveProperty("type");
|
||||||
|
expect(backups[0]).toHaveProperty("size");
|
||||||
|
expect(backups[0]).toHaveProperty("createdAt");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should sort backups by date (newest first)", async () => {
|
||||||
|
// Create backups with different dates by manipulating filenames
|
||||||
|
const oldBackup = path.join(tempDir, "jobs_2026_01_01.db");
|
||||||
|
const newBackup = path.join(tempDir, "jobs_2026_01_15.db");
|
||||||
|
await fs.promises.writeFile(oldBackup, "old");
|
||||||
|
await fs.promises.writeFile(newBackup, "new");
|
||||||
|
|
||||||
|
const backups = await backup.listBackups();
|
||||||
|
|
||||||
|
expect(backups[0].filename).toBe("jobs_2026_01_15.db");
|
||||||
|
expect(backups[1].filename).toBe("jobs_2026_01_01.db");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should ignore non-backup files", async () => {
|
||||||
|
// Create a backup and some other files
|
||||||
|
await backup.createBackup("auto");
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, "random.txt"), "text");
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, "jobs.db"), "db");
|
||||||
|
|
||||||
|
const backups = await backup.listBackups();
|
||||||
|
|
||||||
|
expect(backups).toHaveLength(1);
|
||||||
|
expect(backups[0].filename).toMatch(/^jobs_\d{4}_\d{2}_\d{2}\.db$/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should include suffixed manual backups", async () => {
|
||||||
|
const filename = "jobs_manual_2026_01_01_12_00_00_2.db";
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, filename), "manual");
|
||||||
|
|
||||||
|
const backups = await backup.listBackups();
|
||||||
|
|
||||||
|
expect(backups).toHaveLength(1);
|
||||||
|
expect(backups[0].filename).toBe(filename);
|
||||||
|
expect(backups[0].type).toBe("manual");
|
||||||
|
expect(backups[0].createdAt).toBe("2026-01-01T12:00:00.000Z");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("deleteBackup", () => {
|
||||||
|
it("should delete a backup file", async () => {
|
||||||
|
const filename = await backup.createBackup("auto");
|
||||||
|
const backupPath = path.join(tempDir, filename);
|
||||||
|
|
||||||
|
expect(fs.existsSync(backupPath)).toBe(true);
|
||||||
|
|
||||||
|
await backup.deleteBackup(filename);
|
||||||
|
|
||||||
|
expect(fs.existsSync(backupPath)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should throw error for invalid filename", async () => {
|
||||||
|
await expect(backup.deleteBackup("../../../etc/passwd")).rejects.toThrow(
|
||||||
|
"Invalid backup filename",
|
||||||
|
);
|
||||||
|
await expect(backup.deleteBackup("random.txt")).rejects.toThrow(
|
||||||
|
"Invalid backup filename",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should throw error if backup does not exist", async () => {
|
||||||
|
await expect(backup.deleteBackup("jobs_2026_01_01.db")).rejects.toThrow(
|
||||||
|
"Backup not found",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should delete a suffixed manual backup", async () => {
|
||||||
|
const filename = "jobs_manual_2026_01_01_12_00_00_1.db";
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, filename), "manual");
|
||||||
|
|
||||||
|
await backup.deleteBackup(filename);
|
||||||
|
|
||||||
|
expect(fs.existsSync(path.join(tempDir, filename))).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("cleanupOldBackups", () => {
|
||||||
|
it("should delete oldest automatic backups when exceeding max count", async () => {
|
||||||
|
// Create 7 auto backups (max is 5)
|
||||||
|
for (let i = 1; i <= 7; i++) {
|
||||||
|
const filename = `jobs_2026_01_${String(i).padStart(2, "0")}.db`;
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, filename), "data");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set max count to 5
|
||||||
|
backup.setBackupSettings({ maxCount: 5 });
|
||||||
|
await backup.cleanupOldBackups();
|
||||||
|
|
||||||
|
const remaining = await backup.listBackups();
|
||||||
|
expect(remaining).toHaveLength(5);
|
||||||
|
// Should keep the 5 newest (03-07)
|
||||||
|
const filenames = remaining.map((b) => b.filename);
|
||||||
|
expect(filenames).toContain("jobs_2026_01_03.db");
|
||||||
|
expect(filenames).toContain("jobs_2026_01_07.db");
|
||||||
|
expect(filenames).not.toContain("jobs_2026_01_01.db");
|
||||||
|
expect(filenames).not.toContain("jobs_2026_01_02.db");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should not delete manual backups", async () => {
|
||||||
|
// Create auto backups
|
||||||
|
for (let i = 1; i <= 7; i++) {
|
||||||
|
const filename = `jobs_2026_01_${String(i).padStart(2, "0")}.db`;
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, filename), "data");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create manual backups
|
||||||
|
await fs.promises.writeFile(
|
||||||
|
path.join(tempDir, "jobs_manual_2026_01_01_12_00_00.db"),
|
||||||
|
"manual",
|
||||||
|
);
|
||||||
|
await fs.promises.writeFile(
|
||||||
|
path.join(tempDir, "jobs_manual_2026_01_02_12_00_00.db"),
|
||||||
|
"manual",
|
||||||
|
);
|
||||||
|
|
||||||
|
backup.setBackupSettings({ maxCount: 5 });
|
||||||
|
await backup.cleanupOldBackups();
|
||||||
|
|
||||||
|
const remaining = await backup.listBackups();
|
||||||
|
const autoBackups = remaining.filter((b) => b.type === "auto");
|
||||||
|
const manualBackups = remaining.filter((b) => b.type === "manual");
|
||||||
|
|
||||||
|
expect(autoBackups).toHaveLength(5);
|
||||||
|
expect(manualBackups).toHaveLength(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should not delete if count is within limit", async () => {
|
||||||
|
// Create 3 auto backups (max is 5)
|
||||||
|
for (let i = 1; i <= 3; i++) {
|
||||||
|
const filename = `jobs_2026_01_${String(i).padStart(2, "0")}.db`;
|
||||||
|
await fs.promises.writeFile(path.join(tempDir, filename), "data");
|
||||||
|
}
|
||||||
|
|
||||||
|
backup.setBackupSettings({ maxCount: 5 });
|
||||||
|
await backup.cleanupOldBackups();
|
||||||
|
|
||||||
|
const remaining = await backup.listBackups();
|
||||||
|
expect(remaining).toHaveLength(3);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("setBackupSettings", () => {
|
||||||
|
it("should update settings", () => {
|
||||||
|
backup.setBackupSettings({ enabled: true, hour: 4, maxCount: 3 });
|
||||||
|
|
||||||
|
const settings = backup.getBackupSettings();
|
||||||
|
expect(settings.enabled).toBe(true);
|
||||||
|
expect(settings.hour).toBe(4);
|
||||||
|
expect(settings.maxCount).toBe(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should merge partial settings", () => {
|
||||||
|
backup.setBackupSettings({ hour: 6 });
|
||||||
|
|
||||||
|
const settings = backup.getBackupSettings();
|
||||||
|
expect(settings.enabled).toBe(false); // unchanged
|
||||||
|
expect(settings.hour).toBe(6); // updated
|
||||||
|
expect(settings.maxCount).toBe(5); // unchanged
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("scheduler integration", () => {
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should start scheduler when enabled", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
expect(backup.isBackupSchedulerRunning()).toBe(false);
|
||||||
|
|
||||||
|
backup.setBackupSettings({ enabled: true, hour: 14 });
|
||||||
|
|
||||||
|
expect(backup.isBackupSchedulerRunning()).toBe(true);
|
||||||
|
expect(backup.getNextBackupTime()).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should stop scheduler when disabled", () => {
|
||||||
|
backup.setBackupSettings({ enabled: true, hour: 14 });
|
||||||
|
expect(backup.isBackupSchedulerRunning()).toBe(true);
|
||||||
|
|
||||||
|
backup.setBackupSettings({ enabled: false });
|
||||||
|
expect(backup.isBackupSchedulerRunning()).toBe(false);
|
||||||
|
expect(backup.getNextBackupTime()).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should restart scheduler when hour changes", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
backup.setBackupSettings({ enabled: true, hour: 14 });
|
||||||
|
const firstRun = backup.getNextBackupTime();
|
||||||
|
|
||||||
|
backup.setBackupSettings({ hour: 16 });
|
||||||
|
const secondRun = backup.getNextBackupTime();
|
||||||
|
|
||||||
|
expect(secondRun).not.toBe(firstRun);
|
||||||
|
expect(secondRun).not.toBeNull();
|
||||||
|
expect(firstRun).not.toBeNull();
|
||||||
|
if (secondRun && firstRun) {
|
||||||
|
expect(new Date(secondRun).getTime()).toBeGreaterThan(
|
||||||
|
new Date(firstRun).getTime(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
398
orchestrator/src/server/services/backup/index.ts
Normal file
398
orchestrator/src/server/services/backup/index.ts
Normal file
@ -0,0 +1,398 @@
|
|||||||
|
/**
|
||||||
|
* Database Backup Service
|
||||||
|
*
|
||||||
|
* Manages automatic and manual backups of the SQLite database.
|
||||||
|
* Stores backups in the same directory as the original database.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import fs from "node:fs";
|
||||||
|
import type { FileHandle } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import type { BackupInfo } from "@shared/types.js";
|
||||||
|
import Database from "better-sqlite3";
|
||||||
|
import { getDataDir } from "../../config/dataDir.js";
|
||||||
|
import { createScheduler } from "../../utils/scheduler.js";
|
||||||
|
|
||||||
|
const DB_FILENAME = "jobs.db";
|
||||||
|
const AUTO_BACKUP_PREFIX = "jobs_";
|
||||||
|
const MANUAL_BACKUP_PREFIX = "jobs_manual_";
|
||||||
|
const AUTO_BACKUP_PATTERN = /^jobs_\d{4}_\d{2}_\d{2}\.db$/;
|
||||||
|
const MANUAL_BACKUP_PATTERN =
|
||||||
|
/^jobs_manual_\d{4}_\d{2}_\d{2}_\d{2}_\d{2}_\d{2}(?:_\d+)?\.db$/;
|
||||||
|
|
||||||
|
const AUTO_BACKUP_REGEX = /^jobs_(\d{4})_(\d{2})_(\d{2})\.db$/;
|
||||||
|
const MANUAL_BACKUP_REGEX =
|
||||||
|
/^jobs_manual_(\d{4})_(\d{2})_(\d{2})_(\d{2})_(\d{2})_(\d{2})(?:_\d+)?\.db$/;
|
||||||
|
|
||||||
|
type SqliteDatabase = InstanceType<typeof Database>;
|
||||||
|
|
||||||
|
interface BackupSettings {
|
||||||
|
enabled: boolean;
|
||||||
|
hour: number;
|
||||||
|
maxCount: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Current settings (updated by setBackupSettings)
|
||||||
|
let currentSettings: BackupSettings = {
|
||||||
|
enabled: false,
|
||||||
|
hour: 2,
|
||||||
|
maxCount: 5,
|
||||||
|
};
|
||||||
|
|
||||||
|
// Create scheduler for automatic backups
|
||||||
|
const scheduler = createScheduler("backup", async () => {
|
||||||
|
await createBackup("auto");
|
||||||
|
await cleanupOldBackups();
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the path to the database file
|
||||||
|
*/
|
||||||
|
function getDbPath(): string {
|
||||||
|
return path.join(getDataDir(), DB_FILENAME);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the data directory path
|
||||||
|
*/
|
||||||
|
function getBackupDir(): string {
|
||||||
|
return getDataDir();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Generate filename for a backup
|
||||||
|
*/
|
||||||
|
function generateBackupFilename(type: "auto" | "manual"): string {
|
||||||
|
const now = new Date();
|
||||||
|
if (type === "auto") {
|
||||||
|
// Format: jobs_YYYY_MM_DD.db (UTC date to match UTC scheduler)
|
||||||
|
const year = now.getUTCFullYear();
|
||||||
|
const month = String(now.getUTCMonth() + 1).padStart(2, "0");
|
||||||
|
const day = String(now.getUTCDate()).padStart(2, "0");
|
||||||
|
return `${AUTO_BACKUP_PREFIX}${year}_${month}_${day}.db`;
|
||||||
|
} else {
|
||||||
|
// Format: jobs_manual_YYYY_MM_DD_HH_MM_SS.db (local time for manual backups)
|
||||||
|
const year = now.getFullYear();
|
||||||
|
const month = String(now.getMonth() + 1).padStart(2, "0");
|
||||||
|
const day = String(now.getDate()).padStart(2, "0");
|
||||||
|
const hours = String(now.getHours()).padStart(2, "0");
|
||||||
|
const minutes = String(now.getMinutes()).padStart(2, "0");
|
||||||
|
const seconds = String(now.getSeconds()).padStart(2, "0");
|
||||||
|
return `${MANUAL_BACKUP_PREFIX}${year}_${month}_${day}_${hours}_${minutes}_${seconds}.db`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse backup filename to extract creation date
|
||||||
|
*/
|
||||||
|
function parseBackupDate(filename: string): Date | null {
|
||||||
|
const autoMatch = filename.match(AUTO_BACKUP_REGEX);
|
||||||
|
if (autoMatch) {
|
||||||
|
const [, year, month, day] = autoMatch;
|
||||||
|
return buildUtcDate(year, month, day, "0", "0", "0");
|
||||||
|
}
|
||||||
|
|
||||||
|
const manualMatch = filename.match(MANUAL_BACKUP_REGEX);
|
||||||
|
if (manualMatch) {
|
||||||
|
const [, year, month, day, hours, minutes, seconds] = manualMatch;
|
||||||
|
return buildUtcDate(year, month, day, hours, minutes, seconds);
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildUtcDate(
|
||||||
|
yearRaw: string,
|
||||||
|
monthRaw: string,
|
||||||
|
dayRaw: string,
|
||||||
|
hourRaw: string,
|
||||||
|
minuteRaw: string,
|
||||||
|
secondRaw: string,
|
||||||
|
): Date | null {
|
||||||
|
const year = Number(yearRaw);
|
||||||
|
const month = Number(monthRaw);
|
||||||
|
const day = Number(dayRaw);
|
||||||
|
const hour = Number(hourRaw);
|
||||||
|
const minute = Number(minuteRaw);
|
||||||
|
const second = Number(secondRaw);
|
||||||
|
|
||||||
|
const date = new Date(Date.UTC(year, month - 1, day, hour, minute, second));
|
||||||
|
if (Number.isNaN(date.getTime())) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (
|
||||||
|
date.getUTCFullYear() !== year ||
|
||||||
|
date.getUTCMonth() + 1 !== month ||
|
||||||
|
date.getUTCDate() !== day ||
|
||||||
|
date.getUTCHours() !== hour ||
|
||||||
|
date.getUTCMinutes() !== minute ||
|
||||||
|
date.getUTCSeconds() !== second
|
||||||
|
) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return date;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Determine backup type from filename
|
||||||
|
*/
|
||||||
|
function getBackupType(filename: string): "auto" | "manual" | null {
|
||||||
|
if (AUTO_BACKUP_PATTERN.test(filename)) return "auto";
|
||||||
|
if (MANUAL_BACKUP_PATTERN.test(filename)) return "manual";
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create a backup of the database
|
||||||
|
* @param type - 'auto' for scheduled backups, 'manual' for user-triggered
|
||||||
|
* @returns The filename of the created backup
|
||||||
|
*/
|
||||||
|
export async function createBackup(type: "auto" | "manual"): Promise<string> {
|
||||||
|
const dbPath = getDbPath();
|
||||||
|
const backupDir = getBackupDir();
|
||||||
|
const baseFilename = generateBackupFilename(type);
|
||||||
|
let filename = baseFilename;
|
||||||
|
let backupPath = path.join(backupDir, filename);
|
||||||
|
let reservedHandle: FileHandle | null = null;
|
||||||
|
|
||||||
|
// Check if database exists
|
||||||
|
if (!fs.existsSync(dbPath)) {
|
||||||
|
throw new Error(`Database file not found: ${dbPath}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const tryReserve = async (
|
||||||
|
candidatePath: string,
|
||||||
|
): Promise<FileHandle | null> => {
|
||||||
|
try {
|
||||||
|
return await fs.promises.open(candidatePath, "wx");
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code === "EEXIST") return null;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if (type === "auto") {
|
||||||
|
reservedHandle = await tryReserve(backupPath);
|
||||||
|
if (!reservedHandle) {
|
||||||
|
console.log(
|
||||||
|
`ℹ️ [backup] Auto backup already exists for today: ${filename}`,
|
||||||
|
);
|
||||||
|
return filename;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
const baseName = baseFilename.replace(/\.db$/, "");
|
||||||
|
let sequence = 0;
|
||||||
|
|
||||||
|
while (!reservedHandle && sequence <= 100) {
|
||||||
|
const candidate =
|
||||||
|
sequence === 0 ? baseFilename : `${baseName}_${sequence}.db`;
|
||||||
|
const candidatePath = path.join(backupDir, candidate);
|
||||||
|
const reserved = await tryReserve(candidatePath);
|
||||||
|
if (reserved) {
|
||||||
|
reservedHandle = reserved;
|
||||||
|
filename = candidate;
|
||||||
|
backupPath = candidatePath;
|
||||||
|
} else {
|
||||||
|
sequence += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!reservedHandle) {
|
||||||
|
throw new Error("Failed to create unique manual backup filename");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close the reserved file handle before running SQLite backup
|
||||||
|
await reservedHandle.close();
|
||||||
|
|
||||||
|
let sqlite: SqliteDatabase | null = null;
|
||||||
|
try {
|
||||||
|
sqlite = new Database(dbPath, { readonly: true, fileMustExist: true });
|
||||||
|
await sqlite.backup(backupPath);
|
||||||
|
} catch (error) {
|
||||||
|
await fs.promises.unlink(backupPath).catch(() => undefined);
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
sqlite?.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`✅ [backup] Created ${type} backup: ${filename} (${(await fs.promises.stat(backupPath)).size} bytes)`,
|
||||||
|
);
|
||||||
|
|
||||||
|
return filename;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* List all backups with metadata
|
||||||
|
* @returns Array of backup information
|
||||||
|
*/
|
||||||
|
export async function listBackups(): Promise<BackupInfo[]> {
|
||||||
|
const backupDir = getBackupDir();
|
||||||
|
|
||||||
|
// Check if directory exists
|
||||||
|
if (!fs.existsSync(backupDir)) {
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read directory and filter backup files
|
||||||
|
const files = await fs.promises.readdir(backupDir);
|
||||||
|
const backupFiles = files.filter((file) => {
|
||||||
|
return AUTO_BACKUP_PATTERN.test(file) || MANUAL_BACKUP_PATTERN.test(file);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Get metadata for each backup
|
||||||
|
const backups: BackupInfo[] = [];
|
||||||
|
for (const filename of backupFiles) {
|
||||||
|
const filePath = path.join(backupDir, filename);
|
||||||
|
const type = getBackupType(filename);
|
||||||
|
const createdAt = parseBackupDate(filename);
|
||||||
|
|
||||||
|
if (type && createdAt) {
|
||||||
|
const stats = await fs.promises.stat(filePath);
|
||||||
|
backups.push({
|
||||||
|
filename,
|
||||||
|
type,
|
||||||
|
size: stats.size,
|
||||||
|
createdAt: createdAt.toISOString(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sort by creation date (newest first)
|
||||||
|
backups.sort((a, b) => {
|
||||||
|
return new Date(b.createdAt).getTime() - new Date(a.createdAt).getTime();
|
||||||
|
});
|
||||||
|
|
||||||
|
return backups;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Delete a specific backup
|
||||||
|
* @param filename - Name of the backup file to delete
|
||||||
|
*/
|
||||||
|
export async function deleteBackup(filename: string): Promise<void> {
|
||||||
|
// Validate filename to prevent path traversal
|
||||||
|
if (
|
||||||
|
!AUTO_BACKUP_PATTERN.test(filename) &&
|
||||||
|
!MANUAL_BACKUP_PATTERN.test(filename)
|
||||||
|
) {
|
||||||
|
throw new Error("Invalid backup filename");
|
||||||
|
}
|
||||||
|
|
||||||
|
const backupDir = getBackupDir();
|
||||||
|
const filePath = path.join(backupDir, filename);
|
||||||
|
|
||||||
|
// Check if file exists
|
||||||
|
if (!fs.existsSync(filePath)) {
|
||||||
|
throw new Error(`Backup not found: ${filename}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Delete file
|
||||||
|
await fs.promises.unlink(filePath);
|
||||||
|
console.log(`🗑️ [backup] Deleted backup: ${filename}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clean up old automatic backups
|
||||||
|
* Keeps only the most recent N automatic backups (where N = maxCount)
|
||||||
|
* Manual backups are never deleted automatically
|
||||||
|
*/
|
||||||
|
export async function cleanupOldBackups(): Promise<void> {
|
||||||
|
const backups = await listBackups();
|
||||||
|
|
||||||
|
// Filter to only automatic backups
|
||||||
|
const autoBackups = backups.filter((b) => b.type === "auto");
|
||||||
|
|
||||||
|
// Sort by creation date (oldest first for deletion)
|
||||||
|
autoBackups.sort((a, b) => {
|
||||||
|
return new Date(a.createdAt).getTime() - new Date(b.createdAt).getTime();
|
||||||
|
});
|
||||||
|
|
||||||
|
// Delete oldest backups if we exceed max count
|
||||||
|
const maxCount = currentSettings.maxCount;
|
||||||
|
if (autoBackups.length > maxCount) {
|
||||||
|
const toDelete = autoBackups.slice(0, autoBackups.length - maxCount);
|
||||||
|
|
||||||
|
for (const backup of toDelete) {
|
||||||
|
try {
|
||||||
|
await deleteBackup(backup.filename);
|
||||||
|
} catch (error) {
|
||||||
|
console.error(
|
||||||
|
`❌ [backup] Failed to delete old backup ${backup.filename}:`,
|
||||||
|
error,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`🧹 [backup] Cleaned up ${toDelete.length} old automatic backups (max: ${maxCount})`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Update backup settings and restart scheduler if needed
|
||||||
|
* @param settings - New backup settings
|
||||||
|
*/
|
||||||
|
export function setBackupSettings(settings: Partial<BackupSettings>): void {
|
||||||
|
const oldEnabled = currentSettings.enabled;
|
||||||
|
const oldHour = currentSettings.hour;
|
||||||
|
|
||||||
|
// Update settings
|
||||||
|
currentSettings = { ...currentSettings, ...settings };
|
||||||
|
|
||||||
|
console.log(`⚙️ [backup] Settings updated:`, currentSettings);
|
||||||
|
|
||||||
|
// Restart scheduler if settings changed
|
||||||
|
if (currentSettings.enabled) {
|
||||||
|
if (!oldEnabled || oldHour !== currentSettings.hour) {
|
||||||
|
// Start or restart with new hour
|
||||||
|
scheduler.start(currentSettings.hour);
|
||||||
|
}
|
||||||
|
} else if (oldEnabled && !currentSettings.enabled) {
|
||||||
|
// Stop scheduler
|
||||||
|
scheduler.stop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get current backup settings
|
||||||
|
*/
|
||||||
|
export function getBackupSettings(): BackupSettings {
|
||||||
|
return { ...currentSettings };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Get the next scheduled backup time
|
||||||
|
* @returns ISO string of next backup time, or null if disabled
|
||||||
|
*/
|
||||||
|
export function getNextBackupTime(): string | null {
|
||||||
|
return scheduler.getNextRun();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Check if automatic backup scheduler is running
|
||||||
|
*/
|
||||||
|
export function isBackupSchedulerRunning(): boolean {
|
||||||
|
return scheduler.isRunning();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Start the backup scheduler manually (used on server startup)
|
||||||
|
* Only starts if backup is enabled
|
||||||
|
*/
|
||||||
|
export function startBackupScheduler(): void {
|
||||||
|
if (currentSettings.enabled) {
|
||||||
|
scheduler.start(currentSettings.hour);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stop the backup scheduler
|
||||||
|
*/
|
||||||
|
export function stopBackupScheduler(): void {
|
||||||
|
scheduler.stop();
|
||||||
|
}
|
||||||
@ -184,6 +184,34 @@ export async function getEffectiveSettings(): Promise<AppSettings> {
|
|||||||
: null;
|
: null;
|
||||||
const showSponsorInfo = overrideShowSponsorInfo ?? defaultShowSponsorInfo;
|
const showSponsorInfo = overrideShowSponsorInfo ?? defaultShowSponsorInfo;
|
||||||
|
|
||||||
|
// Backup settings
|
||||||
|
const defaultBackupEnabled = false;
|
||||||
|
const overrideBackupEnabledRaw = overrides.backupEnabled;
|
||||||
|
const overrideBackupEnabled = overrideBackupEnabledRaw
|
||||||
|
? overrideBackupEnabledRaw === "true" || overrideBackupEnabledRaw === "1"
|
||||||
|
: null;
|
||||||
|
const backupEnabled = overrideBackupEnabled ?? defaultBackupEnabled;
|
||||||
|
|
||||||
|
const defaultBackupHour = 2;
|
||||||
|
const overrideBackupHourRaw = overrides.backupHour;
|
||||||
|
const parsedBackupHour = overrideBackupHourRaw
|
||||||
|
? parseInt(overrideBackupHourRaw, 10)
|
||||||
|
: NaN;
|
||||||
|
const overrideBackupHour = Number.isNaN(parsedBackupHour)
|
||||||
|
? null
|
||||||
|
: Math.min(23, Math.max(0, parsedBackupHour));
|
||||||
|
const backupHour = overrideBackupHour ?? defaultBackupHour;
|
||||||
|
|
||||||
|
const defaultBackupMaxCount = 5;
|
||||||
|
const overrideBackupMaxCountRaw = overrides.backupMaxCount;
|
||||||
|
const parsedBackupMaxCount = overrideBackupMaxCountRaw
|
||||||
|
? parseInt(overrideBackupMaxCountRaw, 10)
|
||||||
|
: NaN;
|
||||||
|
const overrideBackupMaxCount = Number.isNaN(parsedBackupMaxCount)
|
||||||
|
? null
|
||||||
|
: Math.min(5, Math.max(1, parsedBackupMaxCount));
|
||||||
|
const backupMaxCount = overrideBackupMaxCount ?? defaultBackupMaxCount;
|
||||||
|
|
||||||
return {
|
return {
|
||||||
...envSettings,
|
...envSettings,
|
||||||
model,
|
model,
|
||||||
@ -242,6 +270,15 @@ export async function getEffectiveSettings(): Promise<AppSettings> {
|
|||||||
showSponsorInfo,
|
showSponsorInfo,
|
||||||
defaultShowSponsorInfo,
|
defaultShowSponsorInfo,
|
||||||
overrideShowSponsorInfo,
|
overrideShowSponsorInfo,
|
||||||
|
backupEnabled,
|
||||||
|
defaultBackupEnabled,
|
||||||
|
overrideBackupEnabled,
|
||||||
|
backupHour,
|
||||||
|
defaultBackupHour,
|
||||||
|
overrideBackupHour,
|
||||||
|
backupMaxCount,
|
||||||
|
defaultBackupMaxCount,
|
||||||
|
overrideBackupMaxCount,
|
||||||
} as AppSettings;
|
} as AppSettings;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -7,6 +7,7 @@
|
|||||||
import fs from "node:fs";
|
import fs from "node:fs";
|
||||||
import path from "node:path";
|
import path from "node:path";
|
||||||
import { getDataDir } from "../../config/dataDir.js";
|
import { getDataDir } from "../../config/dataDir.js";
|
||||||
|
import { createScheduler } from "../../utils/scheduler.js";
|
||||||
|
|
||||||
const DATA_DIR = path.join(getDataDir(), "visa-sponsors");
|
const DATA_DIR = path.join(getDataDir(), "visa-sponsors");
|
||||||
|
|
||||||
@ -492,75 +493,32 @@ export function getOrganizationDetails(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
// Scheduled Updates (Cron-style)
|
// Scheduled Updates (Cron-style) - Uses shared scheduler utility
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
|
|
||||||
let scheduledTimer: ReturnType<typeof setTimeout> | null = null;
|
const scheduler = createScheduler("visa-sponsors", async () => {
|
||||||
let nextScheduledUpdateTime: Date | null = null;
|
await downloadLatestCsv();
|
||||||
|
});
|
||||||
/**
|
|
||||||
* Calculate the next update time (default: 2 AM daily)
|
|
||||||
*/
|
|
||||||
function calculateNextUpdateTime(hour = 2): Date {
|
|
||||||
const now = new Date();
|
|
||||||
const next = new Date(now);
|
|
||||||
next.setHours(hour, 0, 0, 0);
|
|
||||||
|
|
||||||
// If we've passed the time today, schedule for tomorrow
|
|
||||||
if (next <= now) {
|
|
||||||
next.setDate(next.getDate() + 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
return next;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get the next scheduled update time as ISO string
|
* Get the next scheduled update time as ISO string
|
||||||
*/
|
*/
|
||||||
function getNextScheduledUpdate(): string | null {
|
export function getNextScheduledUpdate(): string | null {
|
||||||
return nextScheduledUpdateTime?.toISOString() || null;
|
return scheduler.getNextRun();
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Schedule the next update
|
|
||||||
*/
|
|
||||||
function scheduleNextUpdate(hour = 2): void {
|
|
||||||
if (scheduledTimer) {
|
|
||||||
clearTimeout(scheduledTimer);
|
|
||||||
}
|
|
||||||
|
|
||||||
nextScheduledUpdateTime = calculateNextUpdateTime(hour);
|
|
||||||
const delay = nextScheduledUpdateTime.getTime() - Date.now();
|
|
||||||
|
|
||||||
console.log(
|
|
||||||
`⏰ Next visa sponsor update scheduled for: ${nextScheduledUpdateTime.toISOString()}`,
|
|
||||||
);
|
|
||||||
|
|
||||||
scheduledTimer = setTimeout(async () => {
|
|
||||||
console.log("🔄 Running scheduled visa sponsor update...");
|
|
||||||
await downloadLatestCsv();
|
|
||||||
scheduleNextUpdate(hour); // Schedule the next one
|
|
||||||
}, delay);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Start the scheduler
|
* Start the scheduler
|
||||||
*/
|
*/
|
||||||
export function startScheduler(hour = 2): void {
|
export function startScheduler(hour = 2): void {
|
||||||
console.log("🚀 Starting visa sponsor update scheduler...");
|
scheduler.start(hour);
|
||||||
scheduleNextUpdate(hour);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Stop the scheduler
|
* Stop the scheduler
|
||||||
*/
|
*/
|
||||||
export function stopScheduler(): void {
|
export function stopScheduler(): void {
|
||||||
if (scheduledTimer) {
|
scheduler.stop();
|
||||||
clearTimeout(scheduledTimer);
|
|
||||||
scheduledTimer = null;
|
|
||||||
nextScheduledUpdateTime = null;
|
|
||||||
console.log("⏹️ Stopped visa sponsor update scheduler");
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
195
orchestrator/src/server/utils/scheduler.test.ts
Normal file
195
orchestrator/src/server/utils/scheduler.test.ts
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { calculateNextTime, createScheduler } from "./scheduler.js";
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
vi.useRealTimers();
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("calculateNextTime", () => {
|
||||||
|
it("should return today if hour is in the future", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const result = calculateNextTime(14); // 2 PM
|
||||||
|
|
||||||
|
expect(result.getUTCHours()).toBe(14);
|
||||||
|
expect(result.getUTCDate()).toBe(15); // Same day
|
||||||
|
expect(result.getTime()).toBeGreaterThan(now.getTime());
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should return tomorrow if hour has passed today", () => {
|
||||||
|
const now = new Date("2026-01-15T16:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const result = calculateNextTime(10); // 10 AM (already passed)
|
||||||
|
|
||||||
|
expect(result.getUTCHours()).toBe(10);
|
||||||
|
expect(result.getUTCDate()).toBe(16); // Next day
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should return tomorrow if current time equals target hour", () => {
|
||||||
|
const now = new Date("2026-01-15T14:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const result = calculateNextTime(14); // Same hour
|
||||||
|
|
||||||
|
expect(result.getUTCHours()).toBe(14);
|
||||||
|
expect(result.getUTCDate()).toBe(16); // Next day (since we're at exactly 14:00)
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle hour 0 (midnight)", () => {
|
||||||
|
const now = new Date("2026-01-15T12:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const result = calculateNextTime(0); // Midnight
|
||||||
|
|
||||||
|
expect(result.getUTCHours()).toBe(0);
|
||||||
|
expect(result.getUTCDate()).toBe(16); // Next day
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle hour 23", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const result = calculateNextTime(23);
|
||||||
|
|
||||||
|
expect(result.getUTCHours()).toBe(23);
|
||||||
|
expect(result.getUTCDate()).toBe(15); // Same day
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("createScheduler", () => {
|
||||||
|
it("should create a scheduler with initial stopped state", () => {
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
expect(scheduler.isRunning()).toBe(false);
|
||||||
|
expect(scheduler.getNextRun()).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should start scheduling when start() is called", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
scheduler.start(14);
|
||||||
|
|
||||||
|
expect(scheduler.isRunning()).toBe(true);
|
||||||
|
expect(scheduler.getNextRun()).not.toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should stop scheduling when stop() is called", () => {
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
scheduler.start(14);
|
||||||
|
expect(scheduler.isRunning()).toBe(true);
|
||||||
|
|
||||||
|
scheduler.stop();
|
||||||
|
expect(scheduler.isRunning()).toBe(false);
|
||||||
|
expect(scheduler.getNextRun()).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should execute callback after delay (simulated)", async () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
// Start at hour 10 tomorrow (24 hours from now in this test)
|
||||||
|
scheduler.start(10);
|
||||||
|
|
||||||
|
// Fast-forward time by 24 hours to trigger the callback
|
||||||
|
await vi.advanceTimersByTimeAsync(24 * 60 * 60 * 1000);
|
||||||
|
|
||||||
|
expect(callback).toHaveBeenCalledTimes(1);
|
||||||
|
|
||||||
|
scheduler.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should reschedule after execution", async () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
scheduler.start(10);
|
||||||
|
const firstRun = scheduler.getNextRun();
|
||||||
|
|
||||||
|
// Fast-forward 24 hours to trigger execution and rescheduling
|
||||||
|
await vi.advanceTimersByTimeAsync(24 * 60 * 60 * 1000);
|
||||||
|
|
||||||
|
const secondRun = scheduler.getNextRun();
|
||||||
|
|
||||||
|
// Second run should be 24 hours after first run
|
||||||
|
expect(secondRun).not.toBe(firstRun);
|
||||||
|
expect(secondRun).not.toBeNull();
|
||||||
|
expect(firstRun).not.toBeNull();
|
||||||
|
if (secondRun && firstRun) {
|
||||||
|
expect(new Date(secondRun).getTime()).toBe(
|
||||||
|
new Date(firstRun).getTime() + 24 * 60 * 60 * 1000,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
scheduler.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should restart with new hour when start() called again", () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const callback = vi.fn().mockResolvedValue(undefined);
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
scheduler.start(14); // 2 PM
|
||||||
|
const firstRun = scheduler.getNextRun();
|
||||||
|
|
||||||
|
scheduler.start(16); // 4 PM
|
||||||
|
const secondRun = scheduler.getNextRun();
|
||||||
|
|
||||||
|
// Second run should be later than first run
|
||||||
|
expect(secondRun).not.toBeNull();
|
||||||
|
expect(firstRun).not.toBeNull();
|
||||||
|
if (secondRun && firstRun) {
|
||||||
|
expect(new Date(secondRun).getTime()).toBeGreaterThan(
|
||||||
|
new Date(firstRun).getTime(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
scheduler.stop();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("should handle callback errors gracefully", async () => {
|
||||||
|
const now = new Date("2026-01-15T10:00:00Z");
|
||||||
|
vi.setSystemTime(now);
|
||||||
|
|
||||||
|
const consoleSpy = vi.spyOn(console, "error").mockImplementation(() => {});
|
||||||
|
const callback = vi.fn().mockRejectedValue(new Error("Test error"));
|
||||||
|
const scheduler = createScheduler("test", callback);
|
||||||
|
|
||||||
|
scheduler.start(10);
|
||||||
|
|
||||||
|
// Fast-forward 24 hours to trigger execution
|
||||||
|
await vi.advanceTimersByTimeAsync(24 * 60 * 60 * 1000);
|
||||||
|
|
||||||
|
expect(callback).toHaveBeenCalled();
|
||||||
|
expect(consoleSpy).toHaveBeenCalledWith(
|
||||||
|
"❌ [test] Scheduled task failed:",
|
||||||
|
expect.any(Error),
|
||||||
|
);
|
||||||
|
|
||||||
|
// Scheduler should still be running after error
|
||||||
|
expect(scheduler.isRunning()).toBe(true);
|
||||||
|
|
||||||
|
scheduler.stop();
|
||||||
|
consoleSpy.mockRestore();
|
||||||
|
});
|
||||||
|
});
|
||||||
118
orchestrator/src/server/utils/scheduler.ts
Normal file
118
orchestrator/src/server/utils/scheduler.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* Shared daily scheduler utility for running tasks at a specific hour.
|
||||||
|
* Used by visa-sponsors and backup services.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export interface Scheduler {
|
||||||
|
/** Start scheduling at the specified hour (0-23) */
|
||||||
|
start(hour: number): void;
|
||||||
|
/** Stop the scheduler */
|
||||||
|
stop(): void;
|
||||||
|
/** Get ISO string of next scheduled run, or null if not running */
|
||||||
|
getNextRun(): string | null;
|
||||||
|
/** Check if scheduler is currently running */
|
||||||
|
isRunning(): boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface SchedulerState {
|
||||||
|
timer: ReturnType<typeof setTimeout> | null;
|
||||||
|
nextRunTime: Date | null;
|
||||||
|
currentHour: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Calculate the next occurrence of a specific hour (UTC)
|
||||||
|
* @param hour - Hour of day (0-23) in UTC
|
||||||
|
* @returns Date object set to the next UTC occurrence of that hour
|
||||||
|
*/
|
||||||
|
export function calculateNextTime(hour: number): Date {
|
||||||
|
const now = new Date();
|
||||||
|
const next = new Date(now);
|
||||||
|
next.setUTCHours(hour, 0, 0, 0);
|
||||||
|
|
||||||
|
// If we've passed the time today, schedule for tomorrow
|
||||||
|
if (next <= now) {
|
||||||
|
next.setUTCDate(next.getUTCDate() + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
return next;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Create a reusable daily scheduler
|
||||||
|
* @param name - Service name for logging
|
||||||
|
* @param callback - Async function to execute at scheduled time
|
||||||
|
* @returns Scheduler interface with start/stop/getNextRun methods
|
||||||
|
*/
|
||||||
|
export function createScheduler(
|
||||||
|
name: string,
|
||||||
|
callback: () => Promise<void>,
|
||||||
|
): Scheduler {
|
||||||
|
const state: SchedulerState = {
|
||||||
|
timer: null,
|
||||||
|
nextRunTime: null,
|
||||||
|
currentHour: null,
|
||||||
|
};
|
||||||
|
|
||||||
|
function clearState(): void {
|
||||||
|
if (state.timer) {
|
||||||
|
clearTimeout(state.timer);
|
||||||
|
}
|
||||||
|
state.timer = null;
|
||||||
|
state.nextRunTime = null;
|
||||||
|
state.currentHour = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function scheduleNext(hour: number): void {
|
||||||
|
// Clear any existing timer
|
||||||
|
if (state.timer) {
|
||||||
|
clearState();
|
||||||
|
}
|
||||||
|
|
||||||
|
state.currentHour = hour;
|
||||||
|
state.nextRunTime = calculateNextTime(hour);
|
||||||
|
const delay = state.nextRunTime.getTime() - Date.now();
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`⏰ [${name}] Next run scheduled for: ${state.nextRunTime.toISOString()}`,
|
||||||
|
);
|
||||||
|
|
||||||
|
state.timer = setTimeout(async () => {
|
||||||
|
console.log(`🔄 [${name}] Running scheduled task...`);
|
||||||
|
try {
|
||||||
|
await callback();
|
||||||
|
} catch (error) {
|
||||||
|
console.error(`❌ [${name}] Scheduled task failed:`, error);
|
||||||
|
}
|
||||||
|
// Reschedule for next occurrence
|
||||||
|
scheduleNext(hour);
|
||||||
|
}, delay);
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
start(hour: number): void {
|
||||||
|
if (state.timer) {
|
||||||
|
console.log(`🔄 [${name}] Restarting scheduler with hour ${hour}...`);
|
||||||
|
clearState();
|
||||||
|
} else {
|
||||||
|
console.log(`🚀 [${name}] Starting scheduler at hour ${hour}...`);
|
||||||
|
}
|
||||||
|
scheduleNext(hour);
|
||||||
|
},
|
||||||
|
|
||||||
|
stop(): void {
|
||||||
|
if (state.timer) {
|
||||||
|
clearState();
|
||||||
|
console.log(`⏹️ [${name}] Stopped scheduler`);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
getNextRun(): string | null {
|
||||||
|
return state.nextRunTime?.toISOString() || null;
|
||||||
|
},
|
||||||
|
|
||||||
|
isRunning(): boolean {
|
||||||
|
return state.timer !== null;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
@ -72,6 +72,9 @@ export const updateSettingsSchema = z
|
|||||||
ukvisajobsPassword: z.string().trim().max(2000).nullable().optional(),
|
ukvisajobsPassword: z.string().trim().max(2000).nullable().optional(),
|
||||||
webhookSecret: z.string().trim().max(2000).nullable().optional(),
|
webhookSecret: z.string().trim().max(2000).nullable().optional(),
|
||||||
enableBasicAuth: z.boolean().optional(),
|
enableBasicAuth: z.boolean().optional(),
|
||||||
|
backupEnabled: z.boolean().nullable().optional(),
|
||||||
|
backupHour: z.number().int().min(0).max(23).nullable().optional(),
|
||||||
|
backupMaxCount: z.number().int().min(1).max(5).nullable().optional(),
|
||||||
})
|
})
|
||||||
.superRefine((data, ctx) => {
|
.superRefine((data, ctx) => {
|
||||||
if (data.enableBasicAuth) {
|
if (data.enableBasicAuth) {
|
||||||
|
|||||||
@ -545,4 +545,21 @@ export interface AppSettings {
|
|||||||
ukvisajobsPasswordHint: string | null;
|
ukvisajobsPasswordHint: string | null;
|
||||||
webhookSecretHint: string | null;
|
webhookSecretHint: string | null;
|
||||||
basicAuthActive: boolean;
|
basicAuthActive: boolean;
|
||||||
|
// Backup settings
|
||||||
|
backupEnabled: boolean;
|
||||||
|
defaultBackupEnabled: boolean;
|
||||||
|
overrideBackupEnabled: boolean | null;
|
||||||
|
backupHour: number;
|
||||||
|
defaultBackupHour: number;
|
||||||
|
overrideBackupHour: number | null;
|
||||||
|
backupMaxCount: number;
|
||||||
|
defaultBackupMaxCount: number;
|
||||||
|
overrideBackupMaxCount: number | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface BackupInfo {
|
||||||
|
filename: string;
|
||||||
|
type: "auto" | "manual";
|
||||||
|
size: number;
|
||||||
|
createdAt: string;
|
||||||
}
|
}
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user