Skip to content
Snippets Groups Projects
Commit 9e3a3389 authored by Nicolas Pope's avatar Nicolas Pope
Browse files

More links in readmes

parent c69e845e
No related branches found
No related tags found
No related merge requests found
Pipeline #12958 passed
...@@ -12,7 +12,7 @@ This monorepo contains all elements of the FTL software system. ...@@ -12,7 +12,7 @@ This monorepo contains all elements of the FTL software system.
* [ftl-vision](applications/vision/) : Stereo vision node in p2p network, generates an RGB-Depth net stream * [ftl-vision](applications/vision/) : Stereo vision node in p2p network, generates an RGB-Depth net stream
* [ftl-reconstruct](applications/reconstruct/) : Performs scene reconstruction from synchronised RGB-Depth sources * [ftl-reconstruct](applications/reconstruct/) : Performs scene reconstruction from synchronised RGB-Depth sources
* calibration-multi : All camera intrinsic and extrinsic calibration in one process * calibration-multi : All camera intrinsic and extrinsic calibration in one process
* groupview : A quick camera viewing app that supports frame and video capture * [ftl-view](applications/groupview/) : A quick camera viewing app that supports frame and video capture
* [ftl-gui](applications/gui/) : Desktop GUI * [ftl-gui](applications/gui/) : Desktop GUI
* front-end : Client side FTL code, both web and native * front-end : Client side FTL code, both web and native
* web-service : A web backend service provider acting as a form of proxy * web-service : A web backend service provider acting as a form of proxy
......
...@@ -8,19 +8,20 @@ of socket connections but with a high bandwidth and low latency requirement ...@@ -8,19 +8,20 @@ of socket connections but with a high bandwidth and low latency requirement
in sending and receiving on those sockets. Further work would be needed to in sending and receiving on those sockets. Further work would be needed to
be efficient with large or massive numbers of sockets. be efficient with large or massive numbers of sockets.
The protocol is based on top of MsgPack which works in both C++ and JavaScript. The protocol is based on top of [MsgPack](https://github.com/msgpack/msgpack-c)
To work with JavaScript the protocol works over TCP and TCP+Websockets. The which works in both C++ and JavaScript. To work with JavaScript the protocol
library is also cross platform, supporting Windows and Linux. works over TCP and TCP+Websockets. The library is also cross platform,
supporting Windows and Linux.
It is a template library, allowing simple RPC calls and bindings using the It is a template library, allowing simple RPC calls and bindings using the
latest in C++ features such as optionals, lambdas and futures. latest in C++ features such as optionals, lambdas and futures.
## Universe ## Universe
A Universe class represents a network group and is the primary means of A [Universe class](cpp/include/ftl/net/universe.hpp) represents a network group
interaction for the user of the library. It supports `bind`, `connect`, `call`, and is the primary means of interaction for the user of the library. It supports
`send`, `disconnect`, `asyncCall` and more. `bind`, `connect`, `call`, `send`, `disconnect`, `asyncCall` and more.
## Peer ## Peer
A fairly internal object that wraps a socket connection and deals with all A [Peer object](cpp/include/ftl/net/peer.hpp) is a fairly internal object that
actual sending and receiving over the network. wraps a socket connection and deals with all actual sending and receiving over
the network.
# RGB-Depth Sources # RGB-Depth Sources
This component provides a variety of sources for colour and depth images. These This component provides a variety of sources for colour and depth images. These
include the following: include the following, but do not include virtual cameras:
* Intel Realsense depth camera * [Intel Realsense depth camera](src/realsense_source.hpp)
* Stereo video from two video capture cards using a disparity algorithm * [Stereo video](src/stereovideo.hpp) from two video capture cards using a disparity algorithm
* Snapshots that were previously captured and saved to disk * [Snapshots](src/snapshot_source.hpp) that were previously captured and saved to disk
* Middlebury test datasets as available online * [Middlebury](src/middlebury_source.hpp) test datasets as available online
* Streamed network sources from other nodes * [Streamed network sources](include/ftl/rgbd/streamer.hpp) from other nodes
An RGB-D source is represented as a two image channel object that is generated An RGB-D source is represented as a two image channel object that is generated
through a pipeline of processes usually consisting of the following (but not through a pipeline of processes usually consisting of the following (but not
...@@ -18,29 +18,34 @@ all sources have all steps): ...@@ -18,29 +18,34 @@ all sources have all steps):
## Groups ## Groups
A collection of sources may form a group that must be synchronised accurately A collection of sources may form a group that must be synchronised accurately
for reconstruction to take place. A group class coordinates the above 4 steps for reconstruction to take place. A [group class](include/ftl/rgbd/group.hpp)
across all sources such that millisecond accurate frames with timestamps can coordinates the above 4 steps across all sources such that millisecond accurate
be buffered and collected together to be passed on to the next stage. A high frames with timestamps can be buffered and collected together to be passed on to
precision timer is used to manage the pipeline. the next stage. A [high precision timer](../common/cpp/include/ftl/timer.hpp)
is used to manage the pipeline.
## Streaming ## Streaming
One possible use for a group of sources is to stream them over a network One possible use for a group of sources is to stream them over a network
where they may be re-grouped. A streamer object will receive sets of frames where they may be re-grouped. A [streamer](include/ftl/rgbd/streamer.hpp) object
from a group object and then divide each image into a number of chunks, each will receive sets of frames from a group object and then divide each image into
of which is compressed on a CPU core and sent to every client who asks for them. a number of chunks, each of which is compressed on a CPU core and sent to every
Each client may ask for a different bitrate and resolution so the streamer will client who asks for them. Each client may ask for a different bitrate and
also take care of this. resolution so the streamer will also take care of this. The streamer class uses
the [ftl net library](../net/) for network communication.
## Calibration ## Calibration
Some sources require a camera calibration step. Lens corrections an stereo Some sources require a camera calibration step. Lens corrections an stereo
camera configurations are applied by the calibrate class. Only stereo video camera configurations are applied by the [calibrate class](src/calibrate.hpp).
sources currently need this step. There is also some basic colour correction Only stereo video sources currently need this step and the correction matrices
that can be applied. are calculated using a separate
[calibration app](../../application/calibration-multi/). There is also some
basic [colour correction](src/colour.hpp) that can be applied.
## Disparity Algorithms ## Disparity Algorithms
A few algorithms are included with the RGB-D sources for converting two A few algorithms are included with the RGB-D sources for converting two
colour images into one colour and one depth image based upon the pixel shift colour images into one colour and one depth image based upon the pixel shift
observed between the two images. LibSGM is our algorithm of choice currently. observed between the two images. [LibSGM](https://github.com/fixstars/libSGM)
Further pre and post filtering and smoothing steps are applied, in particular is our algorithm of choice currently. Further pre and post filtering and
an optical flow based temporal smoothing across a number of frames to reduce smoothing steps are applied, in particular an optical flow based temporal
flickering effects. smoothing across a number of frames to reduce flickering effects. This uses
\ No newline at end of file [NVIDIA's Optical Flow SDK](https://developer.nvidia.com/opticalflow-sdk).
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment