AI translation

The command yfm translate can translate documentation using large language models (LLMs). Supported providers are yandexgpt, openai, openrouter, and anthropic.

The pipeline is the same as for other translation providers: text is extracted from the markup, translated, and assembled back. Markdown markup, HTML tags, code, and Liquid constructs do not reach the model — only text segments are translated.

Here, a provider describes an API protocol, not a specific vendor: any compatible installation (self-hosted model, internal gateway) can be connected with the same provider by replacing the API address.

Quick start

1. Get an API key and pass it via an environment variable or the --auth option (a value or a path to a file with the token):

   export OPENAI_API_KEY="sk-..."
   

2. Estimate the translation volume without API requests:

   yfm translate -i . -o ./translated --provider openai --source ru --target en --dry-run
   

   The PROCESSED line will show a forecast of the number of requests and tokens. Files in the output are assembled with the original, untranslated text in this case — do not take them as the translation result.

3. Try translating a single file or section:

   yfm translate -i . -o ./translated --provider openai --source ru --target en \
     --files ru/index.md --cache-dir .translate-cache
   

   Check the quality of the result and, if necessary, configure the glossary or prompts.

4. Run a full pass with caching and asset copying:

   yfm translate -i . -o ./translated --provider openai --source ru --target en \
     --cache-dir .translate-cache --copy-assets
   

5. Check the result: re-running the same command should show requests: 0 — all segments are taken from the cache. The translated version can be built with the usual yfm build.

An error in one file or exceeding limits does not stop the run: failed files are marked with ERR, while the rest continue. Re-running the command will finish the leftovers — already translated segments will be taken from the cache.

Providers

Authorization:

• yandexgpt — IAM token (t1.) or OAuth token (y0_) are passed as Bearer, any other value is passed as the Api-Key of the service account. Additionally, --folder is required — the folder ID, if --model is set with a short name (yandexgpt-lite). The full model URI (gpt://<folder>/yandexgpt/latest) can be specified without --folder.
• openai, openrouter - Bearer key.
• anthropic - key in the x-api-key header.

Connecting compatible installations

A self-hosted model or an internal gateway with a compatible API is connected using the same provider with the --api-base option. The request path is appended to the base automatically:

For openai, openrouter, and anthropic, include /v1 in the base. The base can also be set via the environment variables OPENAI_BASE_URL, OPENROUTER_BASE_URL, ANTHROPIC_BASE_URL.

If the gateway requires its own authorization scheme, pass the headers using the --api-header option (can be repeated). Custom headers override the standard ones, so you can completely replace authorization this way. The --auth option is formally required in this case - pass a placeholder:

yfm translate -i . -o ./translated \
  --provider openai \
  --api-base https://llm.internal.example.com/v1 \
  --model my-model \
  --auth dummy \
  --api-header "Authorization: OAuth $(cat ~/.tokens/llm)" \
  --source ru --target en --cache-dir .translate-cache

The request path for each provider is fixed: if the gateway uses a non-standard path, it cannot be overridden.

Options reference

Common command options (--source, --target, --files, --include, --exclude, --dry-run, and others) are described on the Localization page. The --target option can be passed multiple times - translation will be performed into each language. Below are the AI provider options.

Configuration in a file

All options except --auth can be fixed in the translate section of the configuration file .yfm. Names are in camelCase, command-line flags take precedence:

translate:
  provider: openai
  model: gpt-4o-mini
  cacheDir: .translate-cache
  maxConcurrency: 2
  apiHeaders:
    X-Custom-Header: value

The token cannot be stored in the configuration: the command will fail with the error Do not store authToken in public config. Use environment variables or --auth.

Prompts

The built-in system prompt is tuned for technical translation: preserve markup, do not translate code and identifiers, do not add explanations. You can add your own instructions to it (--prompt-mode append, default) or completely replace it (--prompt-mode replace).

The value of --system-prompt and --user-prompt is a string or a path to a file. Placeholders are supported:

• ``{{source}}, {{target}}` - translation languages;
• ``{{glossary}}` - glossary in text form;
• ``{{context}}` - document context (title and file path);
• ``{{separator}}` - fragment separator;
• ``{{fragments}}, {{text}}- fragments to translate (only in--user-prompt`).

Example: require adherence to a corporate tone:

yfm translate -i . -o ./translated --provider openai --source ru --target en \
  --system-prompt "Use formal tone. Address the reader as 'you'."

Glossary

The model translates each segment separately and does not see how the same term is translated in a neighboring file or in a previous run. Because of this, "сборка" becomes build in one place, assembly in another, and the product name is unexpectedly translated. The glossary sets mandatory translations for terms and eliminates such inconsistency.

Typical cases:

• the product has established terminology, and the translation must match the interface and the rest of the documentation;
• a term, name, or identifier should not be translated at all — then translatedText repeats sourceText;
• the model systematically makes mistakes on a specific term.

The glossary is a YAML file with a single key glossaryPairs. This is a list of pairs "term in the original — required translation":

glossaryPairs:
  - sourceText: оглавление
    translatedText: table of contents
  - sourceText: сборка
    translatedText: build
  - sourceText: Diplodoc
    translatedText: Diplodoc

There is one glossary per run, and it is not tied to a language pair, so for translating into multiple languages you need a separate file for each --target.

The path in --glossary is specified relative to --input:

yfm translate -i . -o ./translated --provider openai --source ru --target en \
  --glossary glossary.yaml

In the configuration file, the path is specified relative to the .yfm itself:

translate:
  glossary: glossary.yaml

If the file is missing, the command will fail with an error.

The pairs are inserted into the prompt of each request to the model as a list of the form term → translation (placeholder {{glossary}}, see Prompts). Three features follow from this:

• This is an instruction to the model, not a text replacement after translation. A term from the glossary is followed almost always, but there is no guarantee: the result should be checked by searching for the translation or by quality assessment.
• The model handles word forms on its own; you do not need to add separate entries for cases and plural forms.
• The entire glossary goes into every request and consumes tokens on each batch. Keep only terms that are truly important or that the model confuses, not the entire product dictionary.

Changing the glossary invalidates the translation cache: after editing the file, all segments are translated again.

Translation cache

The --cache-dir option enables a persistent cache: "segment — translation" pairs are saved to disk, and repeated runs send only new and changed segments to the model. The cache is flushed to disk after each processed file, so interrupting a run is safe — a restart will continue from the same point.

How the cache works:

• For each combination of "provider + model + language pair", a separate file <provider>.<model>.<source>-<target>.json is created. Changing --model does not overwrite the cache of another model, but it does not use it either.
• Changing prompts or the glossary automatically invalidates the cache: saved translations become outdated and are performed again. Updating the CLI with built-in prompts has the same effect.
• --no-cache disables the cache for one run without deleting saved translations.

It makes sense to commit the cache directory to the repository or keep it between CI runs - then, with regular translations, only the changed segments are paid for.

Quality assessment

The --judge option enables translation evaluation by a second model: each "original - translation" pair gets a score from 0 to 100. The mode is strictly optional - token consumption roughly doubles.

yfm translate -i . -o ./translated --provider openai --source ru --target en \
  --cache-dir .translate-cache --judge --judge-model gpt-4o --judge-threshold 80

By default, the same model that performed the translation evaluates it. This is convenient for finding gross errors, but such self-assessment is inflated. For a fair comparison, use --judge-model with a model no weaker than the translating one: a weak judge will not notice the errors of a strong translator.

Results:

• Segments with a score below --judge-threshold are logged as WARN with the score and reason.

• A report translate-quality.<language>.json is written to the output:

  {
    "model": "gpt-4o",
    "threshold": 80,
    "scored": 214,
    "averageScore": 93.4,
    "low": 2,
    "segments": [
      {
        "path": "ru/tools/docs/build.md",
        "source": "Сборка проекта выполняется командой...",
        "translation": "The project is built with...",
        "score": 55,
        "issue": "Omitted the second sentence"
      }
    ]
  }
  

  Only segments below the threshold are included in segments, sorted from worst to best.

• The final line in the log: judge: 214 units scored, average score 93.4/100, 2 below threshold 80. The first number is the count of evaluated segments, not a score.

The evaluation does not affect the translation result and does not interrupt the run: a failure in evaluating an individual batch is logged and skipped. In --dry-run mode, evaluation is not performed.

How to read the log

Troubleshooting

Error 429 (rate limit)

The CLI itself retries the request up to --retry times with an exponential pause and takes the Retry-After header into account. If API limits are still exceeded, restart the run with lower parallelism:

yfm translate -i . -o ./translated --provider openai --source ru --target en \
  --cache-dir .translate-cache --max-concurrency 2

Already translated segments will be taken from the cache; only the remaining ones will go to the model.

WARN Part is too big

The segment turned out to be larger than --max-batch-tokens and remained in the source language. Increase --max-batch-tokens (if necessary, together with --max-output-tokens) or split the text in the source into shorter paragraphs.

Model response was truncated

Errors like response was truncated mean that the model ran out of the response limit. Increase --max-output-tokens or decrease --max-batch-tokens.

File is not translated

If an edit in a file does not make it into the translation, first check the run scope: the --files and --include options narrow the set of files, and changes outside this set do not get into the run — the log for such a file has no TRANSLATE line. This is not a cache issue.

Also keep in mind that the cache is maintained separately for each model: after changing --model, translations from another model are not reused.

Source text in the output

• After --dry-run this is expected: files are assembled without calling the model, with the source text.
• An individual segment may match the original even in a regular run: the model deliberately does not translate text that is already in the target language, proper names, and non-text fragments. An empty model response is never accepted as a translation — in this case, the source text is preserved.

Known limitations

• Pages with ::: page-constructor blocks inside .md files are translated unreliably (translation#273). For now, it is recommended to exclude them from the run via --exclude.
• The request path is fixed for each provider — a gateway with a non-standard API path cannot be connected.
• The model may corrupt inline markup within a segment (links, emphasis). There is no structural Markdown validation after translation — quality assessment helps find such cases.