diff --git a/README.md b/README.md index 809f8fcbda277e1b44312463f0a1c00b1f7a07da..2269fe68c8b670690a14759aa463f343faaf5d0c 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ This monorepo contains all elements of the FTL software system. * [ftl-vision](applications/vision/) : Stereo vision node in p2p network, generates an RGB-Depth net stream * [ftl-reconstruct](applications/reconstruct/) : Performs scene reconstruction from synchronised RGB-Depth sources * calibration-multi : All camera intrinsic and extrinsic calibration in one process - * groupview : A quick camera viewing app that supports frame and video capture + * [ftl-view](applications/groupview/) : A quick camera viewing app that supports frame and video capture * [ftl-gui](applications/gui/) : Desktop GUI * front-end : Client side FTL code, both web and native * web-service : A web backend service provider acting as a form of proxy diff --git a/components/net/README.md b/components/net/README.md index 377eae923cc8ec545c714cf0aa233deb653bd0df..97ceed68e57e0e9441c68e1dbd24e1a18fb300d5 100644 --- a/components/net/README.md +++ b/components/net/README.md @@ -8,19 +8,20 @@ of socket connections but with a high bandwidth and low latency requirement in sending and receiving on those sockets. Further work would be needed to be efficient with large or massive numbers of sockets. -The protocol is based on top of MsgPack which works in both C++ and JavaScript. -To work with JavaScript the protocol works over TCP and TCP+Websockets. The -library is also cross platform, supporting Windows and Linux. +The protocol is based on top of [MsgPack](https://github.com/msgpack/msgpack-c) +which works in both C++ and JavaScript. To work with JavaScript the protocol +works over TCP and TCP+Websockets. The library is also cross platform, +supporting Windows and Linux. It is a template library, allowing simple RPC calls and bindings using the latest in C++ features such as optionals, lambdas and futures. ## Universe -A Universe class represents a network group and is the primary means of -interaction for the user of the library. It supports `bind`, `connect`, `call`, -`send`, `disconnect`, `asyncCall` and more. +A [Universe class](cpp/include/ftl/net/universe.hpp) represents a network group +and is the primary means of interaction for the user of the library. It supports +`bind`, `connect`, `call`, `send`, `disconnect`, `asyncCall` and more. ## Peer -A fairly internal object that wraps a socket connection and deals with all -actual sending and receiving over the network. - +A [Peer object](cpp/include/ftl/net/peer.hpp) is a fairly internal object that +wraps a socket connection and deals with all actual sending and receiving over +the network. diff --git a/components/rgbd-sources/README.md b/components/rgbd-sources/README.md index 6888943a6291cca5f465ec7630ea2fe934a1ee12..90d1d3554c9c4b13044aa3fadc8d3637baa35205 100644 --- a/components/rgbd-sources/README.md +++ b/components/rgbd-sources/README.md @@ -1,12 +1,12 @@ # RGB-Depth Sources This component provides a variety of sources for colour and depth images. These -include the following: -* Intel Realsense depth camera -* Stereo video from two video capture cards using a disparity algorithm -* Snapshots that were previously captured and saved to disk -* Middlebury test datasets as available online -* Streamed network sources from other nodes +include the following, but do not include virtual cameras: +* [Intel Realsense depth camera](src/realsense_source.hpp) +* [Stereo video](src/stereovideo.hpp) from two video capture cards using a disparity algorithm +* [Snapshots](src/snapshot_source.hpp) that were previously captured and saved to disk +* [Middlebury](src/middlebury_source.hpp) test datasets as available online +* [Streamed network sources](include/ftl/rgbd/streamer.hpp) from other nodes An RGB-D source is represented as a two image channel object that is generated through a pipeline of processes usually consisting of the following (but not @@ -18,29 +18,34 @@ all sources have all steps): ## Groups A collection of sources may form a group that must be synchronised accurately -for reconstruction to take place. A group class coordinates the above 4 steps -across all sources such that millisecond accurate frames with timestamps can -be buffered and collected together to be passed on to the next stage. A high -precision timer is used to manage the pipeline. +for reconstruction to take place. A [group class](include/ftl/rgbd/group.hpp) +coordinates the above 4 steps across all sources such that millisecond accurate +frames with timestamps can be buffered and collected together to be passed on to +the next stage. A [high precision timer](../common/cpp/include/ftl/timer.hpp) +is used to manage the pipeline. ## Streaming One possible use for a group of sources is to stream them over a network -where they may be re-grouped. A streamer object will receive sets of frames -from a group object and then divide each image into a number of chunks, each -of which is compressed on a CPU core and sent to every client who asks for them. -Each client may ask for a different bitrate and resolution so the streamer will -also take care of this. +where they may be re-grouped. A [streamer](include/ftl/rgbd/streamer.hpp) object +will receive sets of frames from a group object and then divide each image into +a number of chunks, each of which is compressed on a CPU core and sent to every +client who asks for them. Each client may ask for a different bitrate and +resolution so the streamer will also take care of this. The streamer class uses +the [ftl net library](../net/) for network communication. ## Calibration Some sources require a camera calibration step. Lens corrections an stereo -camera configurations are applied by the calibrate class. Only stereo video -sources currently need this step. There is also some basic colour correction -that can be applied. +camera configurations are applied by the [calibrate class](src/calibrate.hpp). +Only stereo video sources currently need this step and the correction matrices +are calculated using a separate +[calibration app](../../application/calibration-multi/). There is also some +basic [colour correction](src/colour.hpp) that can be applied. ## Disparity Algorithms A few algorithms are included with the RGB-D sources for converting two colour images into one colour and one depth image based upon the pixel shift -observed between the two images. LibSGM is our algorithm of choice currently. -Further pre and post filtering and smoothing steps are applied, in particular -an optical flow based temporal smoothing across a number of frames to reduce -flickering effects. \ No newline at end of file +observed between the two images. [LibSGM](https://github.com/fixstars/libSGM) +is our algorithm of choice currently. Further pre and post filtering and +smoothing steps are applied, in particular an optical flow based temporal +smoothing across a number of frames to reduce flickering effects. This uses +[NVIDIA's Optical Flow SDK](https://developer.nvidia.com/opticalflow-sdk). \ No newline at end of file