Using Content Update Service

The PTV xServer is able to consider dynamic content, such as the current traffic situation. For these features it is required that the content of the PTV xServer can be updated automatically when new data is available. Data, for instance Feature Layer PTV_TrafficIncidents, provided by PTV is published on a data delivery server, which is hosted by PTV, and immediately transferred to one or several xServer via the PTV Content Update Service.

If content updates are enabled the relevant services are notified automatically depending on the content type (e. g. PTV_TrafficIncidents, Basic Toll, Detailed Toll). There is no further configuration needed, which services have to be notified.

Prerequisites

The consumption of content updates requires an installed and running PTV Content Update Service in the local network or on the local machine of the xServer. To update basic toll, detailed toll or traffic incidents a multi-core CPU and 2 GB of RAM are recommended to run the service, to update all types of data simultaneously 3 GB RAM are recommended.

The PTV Content Update Service is shipped as a zip file similar to the PTV xServer and can be set up and started analogously.

The PTV Content Update Service requires a 64-bit Java 8 or Java 17 installation. Although the configuration does allow to use any Java version between 8 and 17, they should not be used as they are not approved by PTV. Furthermore, the environment variable JAVA_HOME has to point to the location of the Java installation (the directory containing the bin directory). If you do not want to change the value of the JAVA_HOME variable, you can use the environment variable CUS2_JAVA_HOME.

Deployment

We recommend to use only one PTV Content Update Service instance to provide data to a whole cluster of PTV xServer in the same local network environment. Usually, all PTV xServer in a cluster use, more or less, the same data. Providing this data via a single PTV Content Update Service therefore reduces Internet traffic significantly.

The PTV Content Update Service must have access to the Internet in order to establish a connection to the PTV Layer Delivery Server which is hosted by PTV.
A proxy between the PTV Content Update Service and the Internet may be configured.
However, a direct Internet connection for the PTV xServer is not required.
Nonetheless, both the PTV Content Update Service and the PTV xServer must be able to reach each other via http(s) without using a proxy server.

On PTV xServer side, the communication with the PTV Content Update Service and the automatic update of dynamic content is handled by the xruntime service, which is deployed with the PTV xServer by default.

Resource Consumption

A running PTV Content Update Service consists of two processes, a wrapper process and a server process, similar to the PTV xServer. The overall resource consumption depends on the amount of data that has to be transferred to consumers.

Therefore it is recommended to deploy the PTV Content Update Service on a machine separate from the PTV xServer.

With a separate deployment, it is ensured that the PTV xServer is not affected in its performance when the amount of data increases, for example during rush hours. By default, the Quartz job scheduler is configured to use 5 parallel threads for downloading data from the Layer Delivery Server. Furthermore, there are several internal worker threads, for example to provide data to consumers or to clean up unused consumer registrations which cannot be changed.

To minimize the effect on a PTV xServer running on the same machine as the PTV Content Update Service, the wrapper process (and also the server process) is configured with a nice-level of 10 by default. This configuration can be changed by altering the PRIORITY variable in the contentupdateservice.sh script.

Data Selection

The selection of data that is downloaded from the PTV Content Update Service is automatically done by the PTV xServer. Based on configured map data and licensed features, the PTV xServer decides which data is needed. For example, if a PTV Europe City Map is configured and the Feature Layer theme PTV_TrafficIncidents is licensed, the PTV xServer will register itself at the PTV Content Update Service requesting all data for traffic incidents in Europe.

Configuration

PTV Content Update Service Configuration

Related file: contentupdateservice.conf

The PTV Content Update Service is configured in the file contentupdateservice.conf which is located in the conf directory of the PTV Content Update Service container.

Several third-party components, such as a database, an ActiveMQ broker or a job scheduler, are used internally in the PTV Content Update Service. Those third-party components may be configured or replaced to some extent, but we recommend to use the default configuration if you do not have a significant reason to change it.

The only required configuration changes are the connection to a data provider, currently always a Layer Delivery Server, which is configured in the core.provider section of the configuration file. The PTV Layer Delivery Server, for example, is configured with the following snippet:


provider {
    ptvgroup-lds {
        rest.url = "https://lds.ptvgroup.com/lds/rest"
        broker.url = "https://lds.ptvgroup.com/ldsbroker"
    }
}

Hint: Currently, only one provider is supported. Please do not configure more than one.

The PTV Content Update Service knows two different ways of communicating with the Layer Delivery Server: subscription to an ActiveMQ topic for required data and polling for new data. None of both mechanisms affects the way data is transferred from the PTV Content Update Service to the xServer.

The kind of mechanism that is used is decided by the setting of the configuration parameter polling in the core.provider section of the configuration file.

Subscription Mechanism

If the subscription mechanism is used, the PTV Content Update Service subscribes itself to a topic at the ldsbroker for the desired kind of data, for example Feature Layer PTV_TrafficIncidents, and then gets notified by the Layer Delivery Server when new data is available. A download job for each new layer is triggered immediately after the notification. This leads to more balanced download traffic, compared to the polling mechanism, but the amount of balance is still depending on the intervals in which new layers are coming in.

To enable the subscription mechanism, the configuration parameter polling has to be set to false. The use of the subscription mechanism requires a broker.url set as described in the example above.

Polling Mechanism

If the polling mechanism is used, the communication between the Layer Delivery Server and the PTV Content Update Service goes the other way round. The PTV Content Update Service polls the Layer Delivery Server for new data in a specific frequency, determined by the given polling interval. Depending on the interval, this mechanism may lead to wave-like downloads because download jobs for several layers are triggered at the same time.

The configuration parameter polling in the core.provider section of the configuration is set to true by default, so no further configuration adjustment is necessary. However, a polling.interval has to be specified. By default, the interval is configured to 120s, like in the following configuration snippet:


provider {
    ptvgroup-lds {
        rest.url = "https://lds.ptvgroup.com/lds/rest"
        polling {
            enabled = true
            interval = 120s
        }
    }
}

Update on demand

By default the PTV xServer gets enabled updates automatically. But it can also be run in an update on demand mode, in which any update can be triggered via xruntime service operation 'triggerContentUpdate' (cf. core.contentupdates.updateMode in the Server Configuration). After a content update has been triggered, the xruntime service operation 'getContentUpdateTriggerStatus' delivers the current state of the content update.

If update on demand mode is enabled for PTV xServer a PTV Content Update Service has to be used that runs also in this mode. To configure the update on demand mode in PTV Content Update Service the parameter updateOnDemand in the core section of the configuration has to be set to true.

Please note: The xruntime service operations 'triggerContentUpdate' and 'getContentUpdateTriggerStatus' can only be used if update on demand mode is enabled for PTV xServer and PTV Content Update Service.

Consumer Limits

Consumer limits can be specified in the core.consumer.registration.limit section. The total number of consumers can be configured with the key total. The PTV Content Update Service will reject registration attempts if this limit has been reached. To further avoid that the total limit is reached due to reconnect attempts of a single xServer, a second limit can be specified which limits the number of registered consumers with the same name. The default settings of these parameters are total = 10 and perName = 2:


registration.limit {
    total = 10
    perName = 2
}

Additional Spring configuration file of PTV Content Update Service: applicationContext.xml

Some basic elements (as the database) of the PTV Content Update Service are configured in the Sping application context file applicationContext.xml, which is located in the conf directory of the PTV Content Update Service container.

Proxy Configuration

To connect to a data provider (i. e. PTV Layer Delivery Server) via a http proxy please add a core.proxy section to the configuration. Currently, basic authentication and NTLM (Windows) is supported.


proxy {
    host = ... // the proxy host
    port = ... //the proxy port
    user = ... // the user name for proxy authentication
    password = ... // the user password for proxy authentication
    ntlm.domain = ... // Windows domain for proxy authentication with NTLM
    workstation = ... // local name of the server that is running PTV Content Update Service (only for NTLM authentication and only necessary, if the PTV Content Update Service cannot detect it automatically)
}

PTV xServer Configuration

Local Machine Setup

Content updates can be enabled separately for Feature Layer and 2 different types of toll data, basic and detailed toll. They are configured in the core.contentupdates.content.featureLayer, core.contentupdates.content.basicToll and core.contentupdates.content.detailedToll sections of the PTV xServer configuration (xserver.conf):


contentupdates {
     content {
         featureLayer {
             enabled = true
         }
         basicToll {
             enabled = true
         }
        detailedToll {
            enabled = true
        }
     }
}

The default configuration sets a default folder which is used to persist updated data. This folder cannot be shared between several server instances but it can be configured as folows:


contentupdates {
        contentUpdatePath = "/absolute/path/to/content-updates"
    }
}

The default configuration assumes a PTV Content Update Service running on the same machine as the PTV xServer using the default ports (50100 resp. 50108).

Local Network Setup

In case that the PTV Content Update Service is running on a different machine within the local network two URLs locating the PTV Content Update Service and broker have to be specified in the core.contentupdates section:


contentupdates {
    contentUpdateService.url = "http://{contentupdateservice-host}:{contentupdateservice-port}/contentupdateservice"
    broker.url = "http://{broker-host}:{broker-port}"
}

It is recommended to deploy the PTV Content Update Service container to a separate machine in a local network to provide data for several PTV xServer running in the same local network.

Data transfer between the PTV Content Update Service and the PTV xServer is done via ActiveMQ broker which is included within PTV Content Update Service container by default. The broker-host is consequently equal to contentupdateservice-host. The remote broker access is preconfigured to use http protocol on the port 50108. This can also be changed by configuring transport connectors within configuration file conf/activemq.xml within the PTV Content Update Service container.

Filtering Dynamic Feature Layer Content

The configuration shipped with the xServer updates dynamic content for all countries included in the map data, but it is possible to restrict the updates to specific Feature Layer countries. At least one theme and provider name have to be specified, otherwise no content will be updated. The default values of all parameters are documented on the ServerConfiguration page.


contentupdates {
    content {
        featureLayer {
            enabled = true
            themes = [ PTV_TrafficIncidents ]
            providerNames = [ tomtom ]
            countries = [ DE, FR, ES ]
        }
    }
}

Configure Toll Updates for Basic Toll

In most cases it is sufficient to set the enabled entry true in the core.contentupdates.content.basicToll section. In addition a mapName and a productName can be set to override the default values. Both are used as filter criteria to request toll data. Detailed information about the default values and when to set these entries can be found in the PTV xServer configuration (xserver.conf). This configuration example sets the mapName explicitly, which dominates the default value retrieved from the map configuration:


contentupdates {
    content {
        basicToll {
            enabled = true
            mapName = "PTV Europe City Map Premium 2017.2H"
        }
    }
}

The following aspects have to be considered when configuring updates for basic toll:

Only one mapName is supported. Otherwise no toll updates will be received.
The used map must not contain any manual toll updates.
A new toll update is downloaded and extracted automatically and is used until a new update is downloaded, even after a server or a module restart.
If the update process recognizes a problem with a new toll update, a rollback to the last applicable update is done. If none is available the toll calculation falls back to the state of the map.

Configure Toll Updates for Detailed Toll

There are 2 additional parameters, providerNames and countries, which have the same meaning as for feature layer updates. In addition a productName can be set to override the default value. All default values are documented on the ServerConfiguration page.


contentupdates {
    content {
        detailedToll {
            enabled = true
            providerNames = [ "here" ]
            countries = [ ]
        }
    }
}

Monitoring

PTV Content Update Service

A binary state of the PTV Content Update Service is provided by its REST-API and is also shown on the dashboard. Furthermore, lists of registered consumers, running jobs, and active message queues can be requested and are displayed on the dashboard, respectively.

PTV xServer

The PTV xServer dashboard contains a dedicated monitoring page for content updates. Detailed information about the update status of each content type is visualized in a corresponding tab page. For example, the tab page for the Feature Layer PTV_TrafficIncidents shows a list of Feature Layer files grouped by country. One should note that multiple files per country might exist during the update process.

Licensing

The PTV Content Update Service is not licensed separately. The data, however, is licensed via the PTV xServer. Without licensed data, for example PTV_TrafficIncidents, the PTV xServer will not be able to register itself at the PTV Content Update Service and therefore no data will be retrieved.