=====================================
Real Time Properties and measurements
=====================================

Introduction
------------

:doc:`IDCP <idcp>` is a Soft Real-Time application. If the internal timing
constraints are not held, no catastrophic failure occurs, just position
measurement and velocity profile control is degraded.

Nevertheless the application *should* perform certain tasks in certain time
frames.

IOC UniDrive communication
--------------------------

.. note::
   The following text only applies to the UniDrive motor control devices. The
   MOCON devices are deprecated, but also use the CAN bus and have a similar
   concept.

The IOC uses a CAN Bus line to communicate with UniDrive motor control devices.
Details of this communication are described in :doc:`uniserv-CAN <uniserv_can>`.

The communication frequency is determined by the CAN Sync object. This is a CAN
Bus message sent by the IOC at a fixed rate. All UniDrive devices receive this
broadcast message at the same time. They reply with their current position and
status. These unicast messages are collected by the IOC. Since the IOC keeps
the precise time stamp when the CAN Sync object was sent, it can assign the
positions and status values to this time even when all the messages arrive a
bit later.

For positions, IDCP implements an *extrapolation correction*. It takes into
account the last known velocity of each axle and uses this with the position
taken at the CAN Sync time to extrapolate the position at the current time.
This compensates for the delay caused by the sequential reception of all
UniDrive positions and the delay for EPICS position measurement record to
process. By this mechanism, the IOC calculates all axle positions for the time
the position measurement record is processed.

Process variables which are named in the following text have the following
variable parts:

[DEVICE]
  This is the device name of the undulator. At the BESSY storage ring, examples
  for device names are: "UE112ID7R" or "U41IT3R".

[VDRIVE]
  This is a single letter that depends on the virtual drive. A virtual drive is
  a group of axles that is moved to one logical destination with a single
  command. Each undulator has a gap drive and many, but not all undulators have
  a shift drive. The VDRIVE part of a PV name is "V" for the gap- and "H" for
  the shift drive ("V" for "vertical", "H" for "horizontal").

.. note:: The position measurement record is [DEVICE]:NdiPr[VDRIVE]PmsCnv.

CAN Sync Object timing
----------------------

The CAN Sync Object is sent by a CAN Bus record which is processed at a fixed rate.

.. note:: Actually record [DEVICE]:AdiAllPmsSynCnt is processed at a fixed
   rate. It processes via EPICS forward link [DEVICE]:AdiAllPmsSynTrg. This
   processes [DEVICE]:AdiUnAllGblPmsCnt which processes
   [DEVICE]:AdiUnAllGblSyn. The last record is the one the sends the CAN Sync
   object.

The CAN Sync interval is defined in the :doc:`IDCP parameter database <idcp_db>`
in parameter `id-data.ID.config.pos_time_interval`. Currently, as of
2026-06-26 it is 0.05s for all undulators except the UE112 and UE56-1
undulator, where it is 0.1s.

The goal here is to send the CAN sync object at a precise time frame with
minimal jitter.

How to measure the CAN Sync Interval
------------------------------------

The device support used by the IOC for CAN Bus communication,
`MultiCAN <https://mcan.sourceforge.io/>`_, provides a function to
get a time stamp when a CAN object was written. This time stamp is
a 64 bit integer with a resolution of one microsecond.

With the Linux implementation of MultiCAN, the time stamp function is
implemented in the
`socan <https://socan.sourceforge.io/>`_ device support.

.. note::
   For the `mvme2100 <https://bitsavers.trailing-edge.com/pdf/motorola/VME/MVME2100/MVME2100_Brochure.pdf>`_
   CPU board, the timestamp doesn't count microseconds but 600/625 microseconds.
   This must be taken into account when calculating times measured on these
   systems.

IDCP presents this timestamp for the CAN Sync object in two EPICS PVs:

- [DEVICE]:NdiPrVPmsCnv.VALU : The upper 32 bits of the time stamp
- [DEVICE]:NdiPrVPmsCnv.VALT : The lower 32 bits of the time stamp

By monitoring these PVs with a simple `camonitor` command like this::

  camonitor [DEVICE]:NdiPrVPmsCnv.VALU [DEVICE]:NdiPrVPmsCnv.VALT > LOGFILE

we can observe the timestamps for longer periods of time.

With `Linux <https://en.wikipedia.org/wiki/Linux>`_ we have a second method of
measuring time stamps by running `candump` from package can-utils.,
assuming that the CAN interface is named "can2"::

  candump -t A can2,080:7FF

Measurements
------------

The statistical evaluation was made for the differences between two CAN Sync
Timestamps. This difference should be 50ms exactly.

All times that follow have the unit seconds (s).

The measurement method is either:

- camonitor: when EPICS PVs with timestamps were used
- candump: when the `candump` program of Linux was used

RTEMS with mvme2100 CPU board
+++++++++++++++++++++++++++++

System properties
:::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
Device                      UE51
operating system            RTEMS
kernel version              4.9
CPU                         mvme2100
=========================  ===============

Measurement over 24 hours
:::::::::::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          camonitor
start of measurement        2026-06-22 12:00:00
end of measurement          2026-06-23 12:00:00
overall time                23:59:59.902206
number of samples/lines     1727993
absolute minimum            0.039830
absolute maximum            0.100109
max. deviation from mean    0.050109
mean                        0.050000
standard deviation          1.761342e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.039500 to 0.040500            1     0.00
0.040500 to 0.041500            0     0.00
0.041500 to 0.042500            0     0.00
0.042500 to 0.043500            0     0.00
0.043500 to 0.044500            0     0.00
0.044500 to 0.045500            0     0.00
0.045500 to 0.046500            0     0.00
0.046500 to 0.047500            0     0.00
0.047500 to 0.048500            0     0.00
0.048500 to 0.049500          465     0.03
0.049500 to 0.050500      1726728    99.93
0.050500 to 0.051500          453     0.03
0.051500 to 0.052500            0     0.00
0.052500 to 0.053500            0     0.00
0.053500 to 0.054500            0     0.00
0.054500 to 0.055500            0     0.00
0.055500 to 0.056500            0     0.00
0.056500 to 0.057500            0     0.00
0.057500 to 0.058500            0     0.00
0.058500 to 0.059500            0     0.00
0.059500 to 0.060500          342     0.02
0.060500 to 0.061500            0     0.00
0.061500 to 0.062500            0     0.00
0.062500 to 0.063500            0     0.00
0.063500 to 0.064500            0     0.00
0.064500 to 0.065500            0     0.00
0.065500 to 0.066500            0     0.00
0.066500 to 0.067500            0     0.00
0.067500 to 0.068500            0     0.00
0.068500 to 0.069500            0     0.00
0.069500 to 0.070500            0     0.00
0.070500 to 0.071500            0     0.00
0.071500 to 0.072500            0     0.00
0.072500 to 0.073500            0     0.00
0.073500 to 0.074500            0     0.00
0.074500 to 0.075500            0     0.00
0.075500 to 0.076500            0     0.00
0.076500 to 0.077500            0     0.00
0.077500 to 0.078500            0     0.00
0.078500 to 0.079500            0     0.00
0.079500 to 0.080500            0     0.00
0.080500 to 0.081500            0     0.00
0.081500 to 0.082500            0     0.00
0.082500 to 0.083500            0     0.00
0.083500 to 0.084500            0     0.00
0.084500 to 0.085500            0     0.00
0.085500 to 0.086500            0     0.00
0.086500 to 0.087500            0     0.00
0.087500 to 0.088500            0     0.00
0.088500 to 0.089500            0     0.00
0.089500 to 0.090500            0     0.00
0.090500 to 0.091500            0     0.00
0.091500 to 0.092500            0     0.00
0.092500 to 0.093500            0     0.00
0.093500 to 0.094500            0     0.00
0.094500 to 0.095500            0     0.00
0.095500 to 0.096500            0     0.00
0.096500 to 0.097500            0     0.00
0.097500 to 0.098500            0     0.00
0.098500 to 0.099500            0     0.00
0.099500 to 0.100500            4     0.00
===================== ============ ========

Measurement over 18 days
::::::::::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          camonitor
start of measurement        2026-06-01 00:00:00
end of measurement          2026-06-19 14:00:00
overall time                18 days, 15:28:24.503000
number of samples/lines     30929062
absolute minimum            0.011896
absolute maximum            0.150005
max. deviation from mean    0.100004
mean                        0.050000
standard deviation          2.256304e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.011500 to 0.012500            1     0.00
0.012500 to 0.013500            1     0.00
0.013500 to 0.014500            1     0.00
0.014500 to 0.015500            0     0.00
0.015500 to 0.016500            1     0.00
0.016500 to 0.017500            0     0.00
0.017500 to 0.018500            2     0.00
0.018500 to 0.019500            0     0.00
0.019500 to 0.020500            3     0.00
0.020500 to 0.021500            0     0.00
0.021500 to 0.022500            0     0.00
0.022500 to 0.023500            0     0.00
0.023500 to 0.024500            0     0.00
0.024500 to 0.025500            0     0.00
0.025500 to 0.026500            0     0.00
0.026500 to 0.027500            0     0.00
0.027500 to 0.028500            0     0.00
0.028500 to 0.029500            0     0.00
0.029500 to 0.030500            0     0.00
0.030500 to 0.031500            0     0.00
0.031500 to 0.032500            0     0.00
0.032500 to 0.033500            0     0.00
0.033500 to 0.034500            0     0.00
0.034500 to 0.035500            0     0.00
0.035500 to 0.036500            0     0.00
0.036500 to 0.037500            0     0.00
0.037500 to 0.038500            0     0.00
0.038500 to 0.039500            3     0.00
0.039500 to 0.040500            5     0.00
0.040500 to 0.041500            0     0.00
0.041500 to 0.042500            0     0.00
0.042500 to 0.043500            0     0.00
0.043500 to 0.044500            0     0.00
0.044500 to 0.045500            0     0.00
0.045500 to 0.046500            0     0.00
0.046500 to 0.047500            1     0.00
0.047500 to 0.048500           24     0.00
0.048500 to 0.049500        12973     0.04
0.049500 to 0.050500     30897546    99.90
0.050500 to 0.051500        12080     0.04
0.051500 to 0.052500           25     0.00
0.052500 to 0.053500            1     0.00
0.053500 to 0.054500            0     0.00
0.054500 to 0.055500            0     0.00
0.055500 to 0.056500            0     0.00
0.056500 to 0.057500            0     0.00
0.057500 to 0.058500            0     0.00
0.058500 to 0.059500            2     0.00
0.059500 to 0.060500         6099     0.02
0.060500 to 0.061500            2     0.00
0.061500 to 0.062500            0     0.00
0.062500 to 0.063500            0     0.00
0.063500 to 0.064500            0     0.00
0.064500 to 0.065500            0     0.00
0.065500 to 0.066500            0     0.00
0.066500 to 0.067500            0     0.00
0.067500 to 0.068500            0     0.00
0.068500 to 0.069500            0     0.00
0.069500 to 0.070500            1     0.00
0.070500 to 0.071500            0     0.00
0.071500 to 0.072500            0     0.00
0.072500 to 0.073500            0     0.00
0.073500 to 0.074500            0     0.00
0.074500 to 0.075500            0     0.00
0.075500 to 0.076500            0     0.00
0.076500 to 0.077500            0     0.00
0.077500 to 0.078500            0     0.00
0.078500 to 0.079500            0     0.00
0.079500 to 0.080500            0     0.00
0.080500 to 0.081500            0     0.00
0.081500 to 0.082500            0     0.00
0.082500 to 0.083500            0     0.00
0.083500 to 0.084500            0     0.00
0.084500 to 0.085500            0     0.00
0.085500 to 0.086500            0     0.00
0.086500 to 0.087500            0     0.00
0.087500 to 0.088500            0     0.00
0.088500 to 0.089500            0     0.00
0.089500 to 0.090500            0     0.00
0.090500 to 0.091500            0     0.00
0.091500 to 0.092500            0     0.00
0.092500 to 0.093500            0     0.00
0.093500 to 0.094500            0     0.00
0.094500 to 0.095500            0     0.00
0.095500 to 0.096500            0     0.00
0.096500 to 0.097500            0     0.00
0.097500 to 0.098500            0     0.00
0.098500 to 0.099500            0     0.00
0.099500 to 0.100500          272     0.00
0.100500 to 0.101500            4     0.00
0.101500 to 0.102500            0     0.00
0.102500 to 0.103500            0     0.00
0.103500 to 0.104500            0     0.00
0.104500 to 0.105500            0     0.00
0.105500 to 0.106500            0     0.00
0.106500 to 0.107500            0     0.00
0.107500 to 0.108500            0     0.00
0.108500 to 0.109500            0     0.00
0.109500 to 0.110500            1     0.00
0.110500 to 0.111500            3     0.00
0.111500 to 0.112500            1     0.00
0.112500 to 0.113500            0     0.00
0.113500 to 0.114500            0     0.00
0.114500 to 0.115500            0     0.00
0.115500 to 0.116500            0     0.00
0.116500 to 0.117500            0     0.00
0.117500 to 0.118500            0     0.00
0.118500 to 0.119500            0     0.00
0.119500 to 0.120500            0     0.00
0.120500 to 0.121500            0     0.00
0.121500 to 0.122500            0     0.00
0.122500 to 0.123500            0     0.00
0.123500 to 0.124500            1     0.00
0.124500 to 0.125500            0     0.00
0.125500 to 0.126500            1     0.00
0.126500 to 0.127500            1     0.00
0.127500 to 0.128500            1     0.00
0.128500 to 0.129500            0     0.00
0.129500 to 0.130500            2     0.00
0.130500 to 0.131500            0     0.00
0.131500 to 0.132500            1     0.00
0.132500 to 0.133500            0     0.00
0.133500 to 0.134500            0     0.00
0.134500 to 0.135500            0     0.00
0.135500 to 0.136500            0     0.00
0.136500 to 0.137500            0     0.00
0.137500 to 0.138500            0     0.00
0.138500 to 0.139500            0     0.00
0.139500 to 0.140500            0     0.00
0.140500 to 0.141500            0     0.00
0.141500 to 0.142500            0     0.00
0.142500 to 0.143500            0     0.00
0.143500 to 0.144500            0     0.00
0.144500 to 0.145500            0     0.00
0.145500 to 0.146500            0     0.00
0.146500 to 0.147500            0     0.00
0.147500 to 0.148500            0     0.00
0.148500 to 0.149500            0     0.00
0.149500 to 0.150500            3     0.00
===================== ============ ========

Linux with Debian 12 (bookworm) and a desktop PC
++++++++++++++++++++++++++++++++++++++++++++++++

System properties
:::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
Device                      UE48
operating system            Linux Debian 12 (bookworm)
kernel version              6.1.76-1
CPU                         Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
=========================  ===============

Measurement
:::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          candump
start of measurement        2026-05-16 12:00:00
end of measurement          2026-05-17 12:00:00
overall time                23:59:59.950126
number of samples/lines     1727999
absolute minimum            0.049405
absolute maximum            0.050572
max. deviation from mean    0.000595
mean                        0.050000
standard deviation          1.151468e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.048500 to 0.049500            7     0.00
0.049500 to 0.050500      1727985   100.00
0.050500 to 0.051500            7     0.00
===================== ============ ========

Linux with Debian 11 (bullseye), Industrial PC regular kernel
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

System properties
:::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
Device                      UE48
operating system            Linux Debian 11 (bullseye)
kernel version              6.1.90-1~bpo11+1
kernel package              linux-image-6.1.0-0.deb11.21-amd64
CPU                         Intel(R) Core(TM) i3-14100
=========================  ===============

Measurement 1 (candump)
:::::::::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          candump
start of measurement        2026-06-17 12:00:00
end of measurement          2026-06-18 12:00:00
overall time                23:59:59.949983
number of samples/lines     1727999
absolute minimum            0.048268
absolute maximum            0.051803
max. deviation from mean    0.001803
mean                        0.050000
standard deviation          3.478053e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.047500 to 0.048500         3310     0.19
0.048500 to 0.049500        82322     4.76
0.049500 to 0.050500      1557417    90.13
0.050500 to 0.051500        82315     4.76
0.051500 to 0.052500         2635     0.15
===================== ============ ========

Measurement 2 (camonitor)
:::::::::::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          camonitor
overall time                23:59:59.899991
number of samples/lines     1727998
absolute minimum            0.048268
absolute maximum            0.051803
max. deviation from mean    0.001803
mean                        0.050000
standard deviation          3.478057e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.047500 to 0.048500         3315     0.19
0.048500 to 0.049500        82318     4.76
0.049500 to 0.050500      1557411    90.13
0.050500 to 0.051500        82311     4.76
0.051500 to 0.052500         2643     0.15
===================== ============ ========

Measurement over 19 days
::::::::::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          camonitor
overall time                19 days, 0:59:58.899987
number of samples/lines     32903978
absolute minimum            0.047673
absolute maximum            0.052580
max. deviation from mean    0.002580
mean                        0.050000
standard deviation          3.484851e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.047500 to 0.048500        60153     0.18
0.048500 to 0.049500      1573348     4.78
0.049500 to 0.050500     29647150    90.10
0.050500 to 0.051500      1575115     4.79
0.051500 to 0.052500        48211     0.15
0.052500 to 0.053500            1     0.00
===================== ============ ========

Linux with Debian 11 (bullseye), Industrial PC realtime kernel
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

System properties
:::::::::::::::::

=========================  ===============
property                    value
=========================  ===============
Device                      UE48
operating system            Linux Debian 11 (bullseye)
kernel version              5.10.257-1
kernel package              linux-image-rt-amd64
CPU                         Intel(R) Core(TM) i3-14100
=========================  ===============

Measurement
:::::::::::

=========================  ===============
property                    value
=========================  ===============
measurement method          camonitor
start of measurement        2026-06-22 12:00:00
end of measurement          2026-06-23 12:00:00
overall time                23:59:59.900000
number of samples/lines     1727998
absolute minimum            0.047862
absolute maximum            0.051794
max. deviation from mean    0.002138
mean                        0.050000
standard deviation          3.520560e-04
=========================  ===============

Histogram Table:

===================== ============ ========
Range                  Count         %
===================== ============ ========
0.047500 to 0.048500         3183     0.18
0.048500 to 0.049500        82489     4.77
0.049500 to 0.050500      1557365    90.13
0.050500 to 0.051500        82095     4.75
0.051500 to 0.052500         2866     0.17
===================== ============ ========

Diskussion
----------

Measurement methods
+++++++++++++++++++

The two measurements of the Linux bullseye system
with the industrial PC and regular kernel show
that the 'candump' and 'camonitor' measurements
give almost identical results. It follows that
measurements can be compared regardless of the
measurement method.

Statistical parameters comparison, 24 hours measurement
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

=========================  ============= ============= ============= ===============
property                    RTEMS         bookworm      bullseye      bullseye(rt)
=========================  ============= ============= ============= ===============
overall time                24 hours      24 hours      24 hours      24 hours
absolute minimum            0.039830      0.049405      0.048268      0.047862
absolute maximum            0.100109      0.050572      0.051803      0.051794
max. deviation from mean    0.050109      0.000595      0.001803      0.002138
mean                        0.050000      0.050000      0.050000      0.050000
standard deviation          1.761342e-04  1.151468e-04  3.478053e-04  3.520560e-04
=========================  ============= ============= ============= ===============

Statistical parameters comparison, 18 days measurement
++++++++++++++++++++++++++++++++++++++++++++++++++++++

=========================  ==========================  =========================
property                    RTEMS                       bullseye
=========================  ==========================  =========================
overall time                18 days, 15:28:24.503000    19 days, 0:59:58.899987
absolute minimum            0.011896                    0.047673
absolute maximum            0.150005                    0.052580
max. deviation from mean    0.100004                    0.002580
mean                        0.050000                    0.050000
standard deviation          2.256304e-04                3.484851e-04
=========================  ==========================  =========================

Histogram comparison, 24 hours measurement
++++++++++++++++++++++++++++++++++++++++++

Here we see how the timing histograms of the tested systems compare. We only
show the counts, not percentages:

===================== ========== ========== ========== =============
Range                  RTEMS      bookworm   bullseye   bullseye(rt)
===================== ========== ========== ========== =============
0.039500 to 0.040500         1          0          0          0
0.040500 to 0.041500         0          0          0          0
0.041500 to 0.042500         0          0          0          0
0.042500 to 0.043500         0          0          0          0
0.043500 to 0.044500         0          0          0          0
0.044500 to 0.045500         0          0          0          0
0.045500 to 0.046500         0          0          0          0
0.046500 to 0.047500         0          0          0          0
0.047500 to 0.048500         0          0       3310       3183
0.048500 to 0.049500       465          7      82322      82489
0.049500 to 0.050500   1726728    1727985    1557417    1557365
0.050500 to 0.051500       453          7      82315      82095
0.051500 to 0.052500         0          0       2635       2866
0.052500 to 0.053500         0          0          0          0
0.053500 to 0.054500         0          0          0          0
0.054500 to 0.055500         0          0          0          0
0.055500 to 0.056500         0          0          0          0
0.056500 to 0.057500         0          0          0          0
0.057500 to 0.058500         0          0          0          0
0.058500 to 0.059500         0          0          0          0
0.059500 to 0.060500       342          0          0          0
0.060500 to 0.061500         0          0          0          0
0.061500 to 0.062500         0          0          0          0
0.062500 to 0.063500         0          0          0          0
0.063500 to 0.064500         0          0          0          0
0.064500 to 0.065500         0          0          0          0
0.065500 to 0.066500         0          0          0          0
0.066500 to 0.067500         0          0          0          0
0.067500 to 0.068500         0          0          0          0
0.068500 to 0.069500         0          0          0          0
0.069500 to 0.070500         0          0          0          0
0.070500 to 0.071500         0          0          0          0
0.071500 to 0.072500         0          0          0          0
0.072500 to 0.073500         0          0          0          0
0.073500 to 0.074500         0          0          0          0
0.074500 to 0.075500         0          0          0          0
0.075500 to 0.076500         0          0          0          0
0.076500 to 0.077500         0          0          0          0
0.077500 to 0.078500         0          0          0          0
0.078500 to 0.079500         0          0          0          0
0.079500 to 0.080500         0          0          0          0
0.080500 to 0.081500         0          0          0          0
0.081500 to 0.082500         0          0          0          0
0.082500 to 0.083500         0          0          0          0
0.083500 to 0.084500         0          0          0          0
0.084500 to 0.085500         0          0          0          0
0.085500 to 0.086500         0          0          0          0
0.086500 to 0.087500         0          0          0          0
0.087500 to 0.088500         0          0          0          0
0.088500 to 0.089500         0          0          0          0
0.089500 to 0.090500         0          0          0          0
0.090500 to 0.091500         0          0          0          0
0.091500 to 0.092500         0          0          0          0
0.092500 to 0.093500         0          0          0          0
0.093500 to 0.094500         0          0          0          0
0.094500 to 0.095500         0          0          0          0
0.095500 to 0.096500         0          0          0          0
0.096500 to 0.097500         0          0          0          0
0.097500 to 0.098500         0          0          0          0
0.098500 to 0.099500         0          0          0          0
0.099500 to 0.100500         4          0          0          0
===================== ========== ========== ========== =============

Histogram comparison, 18 days measurement
+++++++++++++++++++++++++++++++++++++++++

===================== ========== ===========
Range                  RTEMS      bullseye
===================== ========== ===========
0.011500 to 0.012500          1           0
0.012500 to 0.013500          1           0
0.013500 to 0.014500          1           0
0.014500 to 0.015500          0           0
0.015500 to 0.016500          1           0
0.016500 to 0.017500          0           0
0.017500 to 0.018500          2           0
0.018500 to 0.019500          0           0
0.019500 to 0.020500          3           0
0.020500 to 0.021500          0           0
0.021500 to 0.022500          0           0
0.022500 to 0.023500          0           0
0.023500 to 0.024500          0           0
0.024500 to 0.025500          0           0
0.025500 to 0.026500          0           0
0.026500 to 0.027500          0           0
0.027500 to 0.028500          0           0
0.028500 to 0.029500          0           0
0.029500 to 0.030500          0           0
0.030500 to 0.031500          0           0
0.031500 to 0.032500          0           0
0.032500 to 0.033500          0           0
0.033500 to 0.034500          0           0
0.034500 to 0.035500          0           0
0.035500 to 0.036500          0           0
0.036500 to 0.037500          0           0
0.037500 to 0.038500          0           0
0.038500 to 0.039500          3           0
0.039500 to 0.040500          5           0
0.040500 to 0.041500          0           0
0.041500 to 0.042500          0           0
0.042500 to 0.043500          0           0
0.043500 to 0.044500          0           0
0.044500 to 0.045500          0           0
0.045500 to 0.046500          0           0
0.046500 to 0.047500          1           0
0.047500 to 0.048500         24       60153
0.048500 to 0.049500      12973     1573348
0.049500 to 0.050500   30897546    29647150
0.050500 to 0.051500      12080     1575115
0.051500 to 0.052500         25       48211
0.052500 to 0.053500          1           1
0.053500 to 0.054500          0           0
0.054500 to 0.055500          0           0
0.055500 to 0.056500          0           0
0.056500 to 0.057500          0           0
0.057500 to 0.058500          0           0
0.058500 to 0.059500          2           0
0.059500 to 0.060500       6099           0
0.060500 to 0.061500          2           0
0.061500 to 0.062500          0           0
0.062500 to 0.063500          0           0
0.063500 to 0.064500          0           0
0.064500 to 0.065500          0           0
0.065500 to 0.066500          0           0
0.066500 to 0.067500          0           0
0.067500 to 0.068500          0           0
0.068500 to 0.069500          0           0
0.069500 to 0.070500          1           0
0.070500 to 0.071500          0           0
0.071500 to 0.072500          0           0
0.072500 to 0.073500          0           0
0.073500 to 0.074500          0           0
0.074500 to 0.075500          0           0
0.075500 to 0.076500          0           0
0.076500 to 0.077500          0           0
0.077500 to 0.078500          0           0
0.078500 to 0.079500          0           0
0.079500 to 0.080500          0           0
0.080500 to 0.081500          0           0
0.081500 to 0.082500          0           0
0.082500 to 0.083500          0           0
0.083500 to 0.084500          0           0
0.084500 to 0.085500          0           0
0.085500 to 0.086500          0           0
0.086500 to 0.087500          0           0
0.087500 to 0.088500          0           0
0.088500 to 0.089500          0           0
0.089500 to 0.090500          0           0
0.090500 to 0.091500          0           0
0.091500 to 0.092500          0           0
0.092500 to 0.093500          0           0
0.093500 to 0.094500          0           0
0.094500 to 0.095500          0           0
0.095500 to 0.096500          0           0
0.096500 to 0.097500          0           0
0.097500 to 0.098500          0           0
0.098500 to 0.099500          0           0
0.099500 to 0.100500        272           0
0.100500 to 0.101500          4           0
0.101500 to 0.102500          0           0
0.102500 to 0.103500          0           0
0.103500 to 0.104500          0           0
0.104500 to 0.105500          0           0
0.105500 to 0.106500          0           0
0.106500 to 0.107500          0           0
0.107500 to 0.108500          0           0
0.108500 to 0.109500          0           0
0.109500 to 0.110500          1           0
0.110500 to 0.111500          3           0
0.111500 to 0.112500          1           0
0.112500 to 0.113500          0           0
0.113500 to 0.114500          0           0
0.114500 to 0.115500          0           0
0.115500 to 0.116500          0           0
0.116500 to 0.117500          0           0
0.117500 to 0.118500          0           0
0.118500 to 0.119500          0           0
0.119500 to 0.120500          0           0
0.120500 to 0.121500          0           0
0.121500 to 0.122500          0           0
0.122500 to 0.123500          0           0
0.123500 to 0.124500          1           0
0.124500 to 0.125500          0           0
0.125500 to 0.126500          1           0
0.126500 to 0.127500          1           0
0.127500 to 0.128500          1           0
0.128500 to 0.129500          0           0
0.129500 to 0.130500          2           0
0.130500 to 0.131500          0           0
0.131500 to 0.132500          1           0
0.132500 to 0.133500          0           0
0.133500 to 0.134500          0           0
0.134500 to 0.135500          0           0
0.135500 to 0.136500          0           0
0.136500 to 0.137500          0           0
0.137500 to 0.138500          0           0
0.138500 to 0.139500          0           0
0.139500 to 0.140500          0           0
0.140500 to 0.141500          0           0
0.141500 to 0.142500          0           0
0.142500 to 0.143500          0           0
0.143500 to 0.144500          0           0
0.144500 to 0.145500          0           0
0.145500 to 0.146500          0           0
0.146500 to 0.147500          0           0
0.147500 to 0.148500          0           0
0.148500 to 0.149500          0           0
0.149500 to 0.150500          3           0
===================== ========== ===========

Conclusion
----------

RTEMS, not as good as expected
++++++++++++++++++++++++++++++

Although RTEMS has a very good standard deviation of 0.23ms, some extreme
deviations from the mean value are concerning. Over 24 hours we saw a maximum
jitter of 50ms and over 18 days we saw even 100ms. This is longer than the mean
difference between two CAN sync objects. So in the first case, one CAN Sync was
lost, and in the second case even two were lost.

Linux, very good but questions on details remain
++++++++++++++++++++++++++++++++++++++++++++++++

One positive aspect of Linux systems is that the maximum deviation from the
mean value even over 19 days was never greater that 2.6ms.

Some questions remain if we look more closely at the performance:

The real time (rt) kernel of version 5 doesn't perform better than the normal
kernel of version 6. Using the real time kernel makes almost no difference

The bookworm system performed measurably better than the bullseye system, even
when both run with a kernel of version 6. Therefor more tests should be made
with Debian bookworm or even the more recent Debian Trixie.

Linux versus RTEMS
++++++++++++++++++

The measurements suggest that a PC with Linux performs better than the VME Bus
based RTEMS system with a PowerPC CPU regarding the maximum deviations from
the timing schedule.

This indicates that replacing RTEMS VME CPUs with Linux Industrial PCs is the
future for this project.