Checks in CI

Run an originality gate in a pipeline — spin up the engine, submit documents, poll, and fail the build over a threshold.

The engine's API makes a clean CI gate: index your existing content once, then have the pipeline check new documents and fail when similarity crosses a threshold. Useful for content teams (no accidental self-plagiarism across a site), documentation reuse policies, or pre-submission checks.

The gate script

Works against any running engine — a long-lived internal instance or one started in the pipeline:

#!/usr/bin/env bash
set -euo pipefail
 
ENGINE="${ENGINE_URL:-http://localhost:8000}"
THRESHOLD="${MAX_SIMILARITY_PCT:-20}"
FILE="$1"
 
check_id=$(curl -sf -X POST "$ENGINE/v1/checks/upload" \
  -F "file=@$FILE" | jq -r .check_id)
 
for _ in $(seq 1 60); do
  status=$(curl -sf "$ENGINE/v1/checks/$check_id" | jq -r .status)
  [ "$status" = "complete" ] && break
  [ "$status" = "failed" ] && { echo "check failed"; exit 1; }
  sleep 2
done
 
report=$(curl -sf "$ENGINE/v1/checks/$check_id/report")
pct=$(echo "$report" | jq -r .overall_similarity_pct)
 
echo "$FILE: ${pct}% similarity"
if [ "$(echo "$pct > $THRESHOLD" | bc -l)" = "1" ]; then
  echo "$report" | jq -r '.sources[] | "  \(.similarity_pct)%  \(.source_filename // .source_url)"'
  exit 1
fi

GitHub Actions example

Boots the engine as a job container set, indexes the repo's published content, then gates changed files:

jobs:
  originality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Start engine
        run: |
          git clone --depth 1 https://github.com/NoplagLabs/noplag-engine
          cd noplag-engine
          NOPLAG_LOAD_SAMPLE_CORPUS=0 docker compose up -d
          timeout 120 sh -c 'until curl -sf http://localhost:8000/health; do sleep 2; done'
 
      - name: Index existing content
        run: |
          pip install -q httpx
          for f in content/published/*.md; do
            curl -sf -F "file=@$f" http://localhost:8000/v1/corpus/documents
          done
 
      - name: Gate new content
        run: |
          for f in $(git diff --name-only origin/main -- 'content/drafts/*.md'); do
            ./scripts/originality-gate.sh "$f"
          done

Practical notes

Index-then-check ordering: never index the document you're about to check first, or it matches itself at 100%.
Ephemeral vs persistent corpus: re-indexing everything per run (as above) is simple and correct for small corpora. Past a few thousand documents, keep a long-lived engine instance and index on merge instead — see Corpus design patterns.
Thresholds are policy, not truth. A quote-heavy article legitimately scores higher; pick per-content-type thresholds and expect to tune them.
Time budget: very large documents may return coverage: "partial" — decide whether your gate treats that as pass, fail, or retry with a higher NOPLAG_CHECK_BUDGET_S.

The gate script

GitHub Actions example

Practical notes

On this page