So I found a solution. 
Appereanly the OCR prefers white background and black charecters (and not the other way around)
so the following code does the trick:
load image
imshow(TEMP)

BW = ~imbinarize(TEMP);
imshow(BW)

ocrOutput = ocr(BW, 'TextLayout', 'word', 'characterSet', '0123456789')
ocrOutput = 
  ocrText with properties:
                      Text: '297183↵↵'
    CharacterBoundingBoxes: [8×4 double]
      CharacterConfidences: [8×1 single]
                     Words: {'297183'}
         WordBoundingBoxes: [11 12 638 165]
           WordConfidences: 0.9626
                 TextLines: {'297183'}
     TextLineBoundingBoxes: [11 12 638 165]
       TextLineConfidences: 0.9626


