From 06eb3dddccb44bbc1851ceff149c764dad4b06e6 Mon Sep 17 00:00:00 2001 From: Nicolas Pope <nwpope@utu.fi> Date: Mon, 19 Aug 2019 21:08:05 +0300 Subject: [PATCH] Updates to readmes --- README.md | 3 ++ components/net/README.md | 37 +++++++++++++++++-------- components/rgbd-sources/README.md | 46 +++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index caddf61eb..da87d9c35 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,9 @@ This monorepo contains all elements of the FTL software system. * Applications : Executable apps for the different node machines * vision : Stereo vision node in p2p network, generates an RGB-Depth net stream * reconstruct : Performs scene reconstruction from synchronised RGB-Depth sources + * calibration-multi : All camera intrinsic and extrinsic calibration in one process + * groupview : A quick camera viewing app that supports frame and video capture + * gui : Desktop GUI * front-end : Client side FTL code, both web and native * web-service : A web backend service provider acting as a form of proxy * www : FTL Website diff --git a/components/net/README.md b/components/net/README.md index 4c3deb62e..377eae923 100644 --- a/components/net/README.md +++ b/components/net/README.md @@ -1,13 +1,26 @@ # FTL Network Library -FTL Net provides stream, RPC and Peer-2-Peer functionality for the FTL system. -The idea is to allow an efficient mapping to operating system sockets to -minimise userspace copy operations, whilst still allowing for data packing for -smaller RPC calls. The P2P component implements different rpc search strategies -to allow calls to find one, all, many or specific results across the network. - -Multiple protocols are supported, and it is intended that NAT traversal will be -included. However, security, whether encryption or identification, is not -considered presently. - -There are two supported languages: C++ and Javascript. See each respective -language folder for more details. +FTL Net provides an easy to use C++17 network library based around a mix of +streams and RPC. It is now highly optimised to minimise memory copies and +locking, whilst fully taking advantage of all CPU cores for processing +messages received over the network. Each message received is dispatched into +a thread pool. The optimisation works on the basis of a relatively low number +of socket connections but with a high bandwidth and low latency requirement +in sending and receiving on those sockets. Further work would be needed to +be efficient with large or massive numbers of sockets. + +The protocol is based on top of MsgPack which works in both C++ and JavaScript. +To work with JavaScript the protocol works over TCP and TCP+Websockets. The +library is also cross platform, supporting Windows and Linux. + +It is a template library, allowing simple RPC calls and bindings using the +latest in C++ features such as optionals, lambdas and futures. + +## Universe +A Universe class represents a network group and is the primary means of +interaction for the user of the library. It supports `bind`, `connect`, `call`, +`send`, `disconnect`, `asyncCall` and more. + +## Peer +A fairly internal object that wraps a socket connection and deals with all +actual sending and receiving over the network. + diff --git a/components/rgbd-sources/README.md b/components/rgbd-sources/README.md index e69de29bb..6888943a6 100644 --- a/components/rgbd-sources/README.md +++ b/components/rgbd-sources/README.md @@ -0,0 +1,46 @@ +# RGB-Depth Sources + +This component provides a variety of sources for colour and depth images. These +include the following: +* Intel Realsense depth camera +* Stereo video from two video capture cards using a disparity algorithm +* Snapshots that were previously captured and saved to disk +* Middlebury test datasets as available online +* Streamed network sources from other nodes + +An RGB-D source is represented as a two image channel object that is generated +through a pipeline of processes usually consisting of the following (but not +all sources have all steps): +1. Frame capture from hardware +2. Internal buffer swapping if double-buffering is used +3. Retrieval, an IO blocking process of downloading images from devices +4. Computation of, for example, disparity and depth maps from colour images + +## Groups +A collection of sources may form a group that must be synchronised accurately +for reconstruction to take place. A group class coordinates the above 4 steps +across all sources such that millisecond accurate frames with timestamps can +be buffered and collected together to be passed on to the next stage. A high +precision timer is used to manage the pipeline. + +## Streaming +One possible use for a group of sources is to stream them over a network +where they may be re-grouped. A streamer object will receive sets of frames +from a group object and then divide each image into a number of chunks, each +of which is compressed on a CPU core and sent to every client who asks for them. +Each client may ask for a different bitrate and resolution so the streamer will +also take care of this. + +## Calibration +Some sources require a camera calibration step. Lens corrections an stereo +camera configurations are applied by the calibrate class. Only stereo video +sources currently need this step. There is also some basic colour correction +that can be applied. + +## Disparity Algorithms +A few algorithms are included with the RGB-D sources for converting two +colour images into one colour and one depth image based upon the pixel shift +observed between the two images. LibSGM is our algorithm of choice currently. +Further pre and post filtering and smoothing steps are applied, in particular +an optical flow based temporal smoothing across a number of frames to reduce +flickering effects. \ No newline at end of file -- GitLab