DocumentationChangelogUsage guide
Log In
Documentation
Start typing to search…

Diagnostics package (Logs)

Log to file or Log to console:

In the advanced settings ini there are two distinct log output settings:

;enable logging to console log_to_console=0 ;enable logging to file log_to_file=1

Log verbosity:

Within the advanced settings ini file, the level of verbosity of each log can be configured:

Example:

GENERIC=3,3
LICENSE_MANAGER=3,3
FILE_SINK_ELEMENT=3,3
FILE_SRC_ELEMENT=3,3

0=No logging 1= Warning 2= Information 3= Detailed Information 4= Dev level 1 5= Dev level 2

The higher settings can cause high disk usage, recommended levels for customers is 3

Log to file

Log to file uses Live Transcoders logging service. Each element logs to a separate file. The logs are rotated on boot of the host and every 60 minutes. The maximum size by default is 10GB.

If a customer wishes to change the maximum size, they can do this by adding the following environment variable to the docker_compose.yml:

TRC_DISK_LOG_LIMIT_MEGABYTES=

Log to console

Logging output is merged in Journald which is a system service for collecting and storing log data. It creates and maintains structured, indexed journals based on logging information that is received from a variety of sources.

The Journal service stores log data persistently in /var/log/journal

Log to console can be used if the user wishes to send Live Transcoder logging to an external storage location, however the customer must configure journald for this purpose.

Log to file: Diagnostics package important directories and logs

  1. Diagnostics package can be downloaded via the UI: Setup > Diagnostics & Support
  2. This can be downloaded retrospectively and does not need to be downloaded at the time of the issue. However we do recommend if the resolution may be to restart the instance, that the diagnostic package is downloaded before a restart to preserve some of the logs. Logs are kept for 7 days.
  3. Within the diagnostics package there are several useful logs for 1st line support:
    1. report-2023-10-09-10_05_16\version\transcoder
      1. transcoder file contains the transcoder code base version, OS and basic system information
    2. report-2023-10-09-10_05_16\hwinfo
      1. hwinfo contains all hardware information
    3. report-2023-10-09-10_05_16\network\ifconfig
      1. ifconfig contains host network configuration
    4. report-2023-10-09-10_05_16\system\timedatectl
      1. timedatectl contains timezone host is configured in, all transcoder logs correspond to UTC
    5. \report-2023-10-09-10_05_16\log\transcoder*
      1. transcoder directory contains logs for all pipeline and transcoder services the last 24 hours
        1. All previous logs are backed up in the Archive.zip directory
    6. \report-2023-10-09-10_05_16\transcoder.tgz\transcoder\pipelines.json
      1. pipelines.json in this directory is the current pipeline configuration from when the diagnostics package was downloaded
    7. \report-2023-10-09-10_05_16\transcoder.tgz\transcoder\pipelines.json.backup*
      1. This directory contains all previous pipelines configuration jsons
    8. \report-2023-10-09-10_05_16\transcoder.tgz\transcoder\settings\settings.ini
  4. XXXX-XX-XX_XX_MACHINE_STATUS_Controller_XX.log - Contains CPU, GPU, Network utilization
  5. XXXX-XX-XX_XX_UI_XX.log - Contains UI activity
  6. XXXX-XX-XX_XX_GENERIC_Controller_XX.log - Contains systemctl activity
  7. XXXX-XX-XX_XX_TS_DEMUX_ELEMENT_Demux_XX_XX_XXX.log - Contains activity per pipeline regarding demux
  8. XXXX-XX-XX_XX_TS_MUX_ELEMENT_TsMux_XX_XX_XXX.log - Contains activity per pipeline regarding regarding muxer

Which logs relate to which pipeline components?

Transcoder logs are found in: \report_live-transcoder-YYYY-MM-DD-HH_MM_SS.tar\log.zip\log\transcoder\transcoder\

YYYY-MM-DD_HH_XX_Demux_0_0_XXXX.log logs relate to the input of the pipeline. The suffix of the filename maps the file to a particular pipeline and input. i.e. Pipeline ID:0, Source 0 in the example given

The same is the case for the output of the pipeline, YYYY-MM-DD_HH_mux_0_0_XXXX.log would be equal to Pipeline ID:0, Output 0

The last digits describe the process ID as managed in the OS. Each time the pipeline restarts OR stops/starts, each process will start again, therefore each element will assume a new process ID.


Examples of common errors, statistics and warnings:

MPEG-TS (SRT, UDP, RTP)

  1. Demuxer has detected a continuity counter error, either the a count has been skipped or duplicated. This could happen during the encoding or transport of the stream, recommended to inspect the stream origin, and any devices/links it may transit through. A third party device that can also detect CC errors is useful for determining the origin of the error:
Nov 25 11:27:12 transcoder0 transcoder-base[772]: 11:27:12.413  TS_DEMUX_ELEMENT        PidHandler.cpp:31      Demux_0_0(772)    H264 elementary stream pid: 101 Detected packet drop (mismatch in CC)
  1. TS muxer is dropping a packet, this could be that the buffer has been starved due to issues on the input, there was not enough compute resource to create the packet or there was not enough time relative to the PTS to create/send the packet:
Nov 25 13:51:35 transcoder0 transcoder-base[609]: 13:51:35.188  TS_MUX_ELEMENT          TsMuuxElement.cpp:1023     TsMux_1_0(609)    PID: 101 drop packets: 94
  1. TS demuxer reports PCR from source, Latest PCR (27MHz) = PTS time, a = current, b = offset
Nov 25 13:51:35 transcoder0 transcoder-base[589]: 13:51:35.212  TS_DEMUX_ELEMENT        PcrState.cpp:501     Demux_0_0(589)    PCR report:  Latest PCR (27MHz): 291811910657 [ a = 0.999932 , b = 2.90792e+11 ] Discontinuities: 0 PCR-day: 0

SRT:

SRT pipeline configuration:

Nov 25 10:52:09 transcoder0 transcoder-base[385]: 2024-11-25 10:52:09 [http-nio-127.0.0.1-8080-exec-2] INFO  c.c.t.ui.type.ConfigurationSession - PIPELINE_MODEL_APPLIED: Pipeline (id:64, name: Pipeline) inputs:  source: udp://0.0.0.0:5100 srt: Srt{mode=LISTENER, latency=240, overhead=20, streamID=, isEncrypted=false, encryption=AES_DEFAULT, passphrase=}

SRT Receiver (Demuxer) in Listener mode:

Nov 25 10:52:49 transcoder0 transcoder-base[762]: 10:52:49.598  TS_DEMUX_ELEMENT        TsDemuxElement.cpp:279     Demux_0_0(762)    SRT Listening

SRT Listener is functioning correctly, however no device is currently calling to open connection on the address(es) and port specified, can also be related to start of pipeline or continuous packet loss

Nov 25 11:00:42 transcoder0 transcoder-base[322]: 11:00:42.547603/CMPTO_NET_INIT*E:SRT.cn: srt_accept: no pending connection available at the moment
Nov 25 11:01:57 transcoder0 transcoder-base[954]: 11:01:57.611  TS_MUX_ELEMENT          Sender.cpp:212     TsMux_1_0(954)    SRT connection failed. Trying again. srt://127.0.0.1:5050:lo Error:  5

SRT NAK

Nov 25 11:26:49 transcoder0 transcoder-base[772]: 11:26:49.906758/SRT:TsbPd!W:SRT.br: @910563377: RCV-DROPPED 2 packet(s). Packet seqno %900016611 delayed for 0.084 ms
Nov 25 11:26:15 transcoder0 transcoder-base[772]: 11:26:15.345  TS_DEMUX_ELEMENT        PidHandler.cpp:31      Demux_0_0(772)    H264 elementary stream pid: 101 Detected packet drop (mismatch in CC)11:26:15.345412/SRT:TsbPd!W:SRT.br: @910563377: RCV-DROPPED 1 packet(s). Packet seqno %899988020 delayed for 0.452 ms

NDI:

Start of NDI Output stream:

Nov 25 10:58:17 transcoder0 transcoder-base[92]: 10:58:17.319  NDI_SINK_ELEMENT        NdiSinkElement.cpp:595     Controller(92)    Adding video stream to NDI Sink element.

NDI Receiver starts receiving NDI video:

Dec 05 13:35:31 transcoder0 transcoder-base[1142]: 13:35:31.468  NDI_SRC_ELEMENT         NdiReceiver.cpp:52      NdiSrc_13_0(1142)    Receiving audio/video with frame rate: 50

This line reports the transcode latency in milli-seconds, reporting average and maximum latency during x period:

Nov 25 11:01:40 transcoder0 transcoder-base[780]: 11:01:40.341  LATENCY_MEASUREMENT     NdiSinkElement.cpp:524     NdiMux_0_0_0(780)    NdiMux_0_0_0 Latency: 2810.88 avg, 2864.5 max

Start of NDI Input stream:

Dec 05 13:35:27 transcoder0 transcoder-base[1142]: 13:35:27.642  NDI_SRC_ELEMENT         NdiSourceElement.cpp:133     NdiSrc_13_0(1142)    Looking for NDI source 'TRANSCODER0 (zz_12345)'

NDI_sink_element (NDI Output) has encountered a missing frame from the input, PTSJitterRemover logs this message as a warning if this occurs, PTS expected value is larger than 1800/90KHz = 20ms or a single frame for 50Hz video :

Dec 05 13:38:51 transcoder0 transcoder-base[1140]: 13:38:51.345  NDI_SINK_ELEMENT        PtsJitterRemover.cpp:33      NdiMux_12_0_0(1140)    PTSJitterRemover resetting due to diff > maxPTSJitter: 1800 > 9

Updated 5 months ago