Discovery Configuration

What Discovery Does

The Discovery Engine continuously monitors 31+ trend sources and feeds every signal into a scoring pipeline tuned to your client's brand context. Each raw signal (a rising Reddit thread, a spiking search query, a breakout TikTok hashtag) is evaluated against four weighted criteria: trend velocity, brand relevance, competition gap, and historical engagement. Signals that pass the account's configured score threshold are promoted to content buckets and immediately surfaced to the Production pipeline. Everything below the threshold is stored in the global trend index for retrospective review. No manual triage required.

Source Types

Discovery ingests six categories of source. Each type has a different data shape and refresh cadence:

• RSS feeds, any standard RSS 2.0 or Atom feed. Polled every 15 minutes. Suitable for news sites, industry blogs, and competitor publication feeds.
• Reddit subreddits, post and comment velocity from nominated subreddits via the Reddit public API. Polled every 30 minutes. Useful for community sentiment and emerging niche topics.
• X / Twitter searches, keyword and hashtag searches via the X API v2. Polled every 5 minutes for high-priority pipelines, every 15 minutes for standard. Best for real-time breaking topics.
• Google Trends, interest-over-time and related-query data for up to 5 simultaneous keywords per pipeline. Refreshed hourly. Use this to validate velocity before committing production resources.
• TikTok hashtags, view count and post velocity for nominated hashtags via the TikTok Research API. Polled every 15 minutes. High signal-to-noise for short-form video strategy.
• Custom URLs, any publicly accessible page. The crawler extracts text using either a CSS selector you provide or the page's detected feed URL. Polled on the schedule you configure (minimum 15 minutes).

Adding a Source

Navigate to Discovery → Sources → Add Source in the client account. Select the source type and fill in the type-specific fields:

• RSS: feed URL only. The engine auto-detects format (RSS 2.0, Atom, JSON Feed).
• Reddit: subreddit name without the r/ prefix (e.g. marketing). Optionally set a minimum upvote threshold to suppress low-signal posts.
• X / Twitter: a search query string using standard X search syntax, including hashtags, operators, and language filters (e.g. content marketing lang:en -is:retweet).
• Google Trends: one or more keywords. Optionally scope to a geographic region and time window.
• TikTok hashtag: hashtag text without the # (e.g. contentcreator).
• Custom URL: the target URL plus either a CSS selector targeting the content container (e.g. article.post-body) or the page's feed URL if one exists. If neither is supplied, the engine falls back to full-page text extraction, which is noisier.

After saving, Discovery runs an immediate test fetch and displays the first 5 extracted signals for review. If the test fetch returns zero results, check the source configuration before enabling it.

Scoring Weights

Every signal is scored 0–100 using four dimensions. Default weights reflect a general-purpose content strategy; adjust them in Discovery → Scoring → Edit Weights per pipeline:

• Relevance (brand fit), default 30 %. Measures how closely the topic aligns with the client's defined niche, keywords, and competitor set. Raise this weight for highly niche accounts where off-topic content is harmful to brand consistency.
• Recency, implicit in velocity; signals decay exponentially after 48 hours. Not a standalone slider but factored into the velocity score automatically.
• Authority, default 20 %. Measures the source's domain authority, follower count, or subreddit subscriber base. Downweight this for client niches where micro-communities are strategically important.
• Engagement, default 40 %. Upvotes, shares, replies, and saves normalized by source median. This is the strongest predictor of content performance and is rarely worth reducing below 25 %.

Weights must sum to 100. The UI enforces this constraint and will not save an invalid configuration.

Filters

Filters run before scoring and can significantly reduce noise. Configure them in Discovery → Filters:

• Blocklist keywords: exact strings or simple globs (e.g. *crypto*). Any signal whose title or body matches a blocklist entry is dropped before scoring.
• Allowlist phrases: signals that do not contain at least one allowlist phrase are suppressed. Leave blank to disable. Useful when you want Discovery restricted to a very specific topic cluster.
• Language: ISO 639-1 code (e.g. en, es). Signals in other languages are dropped. Defaults to the account's primary language setting.
• Region: ISO 3166-1 alpha-2 country code. Only applies to sources that carry geographic metadata (Google Trends, some X searches). Leave blank for global signals.

Refresh Cadence

Each source type has a minimum and default polling interval. You can slow down any source to reduce API consumption, but cannot poll faster than the minimums:

• RSS: default 15 min, minimum 15 min
• Reddit: default 30 min, minimum 15 min
• X / Twitter: default 15 min (standard), 5 min (high-priority)
• Google Trends: default 60 min, minimum 60 min
• TikTok hashtag: default 15 min, minimum 15 min
• Custom URL: configurable, minimum 15 min

High-priority polling (5 min) for X is available on Pro plans and above. Standard plans default to 15 min for all sources.

Trend Scout Agent

The Trend Scout is the AI agent that runs on top of the Discovery Engine and synthesizes cross-source signals into a ranked opportunity brief. It runs after every aggregation cycle, groups related signals by theme, and writes a short rationale for each top-ranked topic. See Platform Overview for how Trend Scout fits into the full content pipeline.

Common Gotchas

• Empty discovery results. The most common cause is allowlist phrases that are too narrow, or a blocklist that over-matches. Start with no allowlist and a minimal blocklist, confirm signals flow, then tighten incrementally.
• Trending vs. evergreen tradeoff. Heavy weighting toward engagement rewards fast-moving topics and suppresses slower evergreen content. If your client needs a consistent baseline of evergreen output, either create a separate pipeline with lower engagement weight or manually promote evergreen signals from the global trend index.
• Custom URL selector drift. Pages sometimes restructure their HTML, silently breaking your CSS selector. Monitor the Last successful fetch timestamp on each custom source and set an alert if it exceeds 2x the configured polling interval.
• X rate limits. The X API v2 Basic tier allows 500,000 posts read per month per app. Each high-priority source consumes roughly 2,000 posts per day. Plan source counts accordingly.