** Notice: all internal pipelines have been restricted to 30 minute timeouts, if your CI/CD runs take longer, either use your own runners or read below **
We would remind everyone that the CI/CD services provided by the university Gitlab servers are intended for CI/CD use only (ie. delivering new versions of your software to your production servers, etc).They are not intended for data analysis or -production use, or creating, building and running servers to do heavy data calculation or visualization.
As a general guideline; if your jobs take more than 15 minutes, you might be running something that is better suited for a dedicated system or a virtual server - in this case, please contact the helpdesk for a solution to your needs.
[Redis streams](https://redis.io/docs/data-types/streams-tutorial/) are used to inform about update events, typically "create", "delete" and "update" for various resource types. Relevant information about the state changes is included with the event. Events are sent using the `redisSendEvent` function defined in `@ftl/common` package.
The owner of events is the generator of that event, that services defines the structures inside a corresponding directory of the `@ftl/api` package. The API package should only contain typescript definitions for the events, no actual Javascript code.
The owner of events is the generator of that event, that service defines the structures inside a corresponding directory of the `@ftl/api` package. The API package should only contain typescript definitions for the events, no actual Javascript code.
Events usually include common information such as session, request and user identifiers so that the origin of the event can be traced. Typically, each general type of event has its own Redis stream key (eg. `events:stream`), but there might be subcategories of event within that stream such as `start` / `stop` or `create` / `destroy`.
Events usually include common information such as session, request and user identifiers so that the origin of the event can be traced. Typically, each general type of event has its own Redis stream key (eg. `events:node`), but there might be subcategories of event within that stream such as `start` / `stop` or `create` / `destroy`.
Other services can this listen for these events by registering a callback with `redisSetStreamCallback`. Each service type is a "consumer group" which means that only one instance of that service type will receive the event, not all instances. The idea is that the receiving instance will make some database update or take some action and that other instances of the same service can become aware of the consequences by some other means.
Other services can then listen for these events by registering a callback with `redisSetStreamCallback`. Each service type is a "consumer group" which means that only one instance of that service type will receive the event, not all instances. The idea is that the receiving instance will make some database update or take some action and that other instances of the same service can become aware of the consequences by some other means.
If some unhandled exception occurs in the event callback, or if the service crashes before completing it, then the original event is kept as pending and unprocessed by Redis. Only upon successful completion of the callback is the event removed from the queue for that service type.
# Commands
Command streams instruct other services to perform some action. These are the inverse of an event and are owned by the service which responds to the command. Using events requires that a service becomes dependent and knowledgeable about the service generating the event, whereas using commands it need not know about other services who may issue the command. Keeping inter-service dependencies to a minimum and directed (a directed rather than cyclic graph of dependencies) is important.
