Cognitive Services-Translator Text: Uploading Dictionary Set Fails Due to Sentence Count Mismatch

Hello @Søren Printz Christensen ,

This error means Custom Translator found a different number of sentences in your source file than in your target file. Dictionary entries are paired one-to-one: line 1 of the source with line 1 of the target, and so on. If one entry gets split in two on one side, the counts no longer match, even when both files have the same number of lines.

Since you've already checked line counts, encoding and blank lines, please try these steps:

1. Check for punctuation inside an entry. The service splits text into sentences. An entry with ., ? or ! in the middle (for example Dr. Smith or Ready? Go) can be counted as two sentences on one side only. Check carefully for stray characters too. A single ? that appears when a file is converted or saved in another encoding can cause this.
2. Rename both files to use the .align extension (for example glossary_en.align and glossary_de.align) and upload again. With .align, Custom Translator treats the files as already aligned and doesn't do its own sentence alignment.
3. Or use one Excel (.xlsx) file. Put the source language code in A1 and the target code in B1. Then add one entry per row below them, with no line breaks inside cells.
4. Find the problem line. This script flags hidden line breaks, punctuation inside an entry, and entries that are empty on only one side:

import re, sys
def load(p):
    raw = open(p, 'rb').read()
    enc = 'utf-16' if raw[:2] in (b'\xff\xfe', b'\xfe\xff') else 'utf-8-sig'
    return raw.decode(enc).replace('\r\n', '\n').rstrip('\n').split('\n')
src, tgt = load(sys.argv[1]), load(sys.argv[2])
print(f"source: {len(src)} lines | target: {len(tgt)} lines")
hidden = re.compile('[\u2028\u2029\u0085\x0b\x0c\r]')
midstop = re.compile(r'[.!?]\s+\S|[。！？]\s*\S')
for i, (a, b) in enumerate(zip(src, tgt), 1):
    for side, line in (('source', a), ('target', b)):
        if hidden.search(line):
            print(f"line {i} ({side}): hidden line break")
        if midstop.search(line):
            print(f"line {i} ({side}): punctuation inside entry -> {line!r}")
    if bool(a.strip()) != bool(b.strip()):
        print(f"line {i}: empty on one side only")

Run it with: python check.py source.txt target.txt

Fix the flagged lines in both files, or remove the punctuation, and upload again. If the script doesn't flag anything, split the dictionary in half and upload each half. Keep halving the half that fails until you find the bad entry.

If it still fails, please reply with the script output, the file names and extensions, the language pair, and whether you selected Phrase or Sentence dictionary.

Reference: Custom Translator document formats and naming conventions

If this helps, please click Accept Answer so others with the same error can find it.