Developer Update: EVO-3250

This document outlines the recent changes regarding media handling, API request scheduling, and post deletion detection for Instagram and TikTok integrations.

1. On-Demand Media Resolution (Replacing S3 Storage)

Previously, media assets for published posts were stored in S3. This caused storage bloat and sync issues. Now, media URLs are resolved dynamically from the Instagram and TikTok APIs when needed, using temporary CDN URLs.

Key Changes:

Added a new core service (mediaUrlService.js) to summarize and centralize media handling.
S3 media processing for external posts has been removed.
When retrieving posts through the API, dynamic URLs are automatically injected using methods like injectMediaUrlsForPost.
A built-in TTL cache prevents excessive API calls.

Media URL Resolution Flow

2. Intelligent Fetch Backoff

To reduce API rate limit issues, the system now uses dynamic intervals to fetch insights (analytics) rather than querying at a constant flat rate.

Key Changes:

Background jobs (Cron Schedulers) adjust the fetch frequency based on a post’s engagement velocity.
New posts or viral posts are checked frequently.
Older, inactive posts are checked less often (using exponential backoff between 24 and 720 hours).
Database fields nextInsightsFetchAt, insightsFetchIntervalHours, and lastInteractionCount manage this schedule.

3. Native Deletion Detection

The system can now detect when a creator deletes a post directly on the TikTok or Instagram apps.

Key Changes:

Schedulers compare API responses against our local database in the background.
If a post is missing from the general feed, a direct API call verifies its specific status.
If confirmed deleted, the post is marked as deleted_on_platform in the database.
These posts are automatically excluded from overall performance analytics and viral alerts.

Sync and Deletion Detection Flow

Developer Action Items

Media Handling: Do not assume media URLs are stored natively in the database for published posts. Ensure any new endpoints returning scheduled posts route data through mediaUrlService to inject CDN links.
Filtering Deleted Posts: If writing new queries that calculate overall insights or trigger alerts, ensure you explicitly filter out posts with the deleted_on_platform status to prevent dirty data.
Testing Insights Locally: If a post is not updating during local testing via the cron endpoint, it may be paused by the backoff mechanism. Check the nextInsightsFetchAt timestamp in the database to verify if it is eligible for a fetch.