Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
F
ftl
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Nicolas Pope
ftl
Commits
9e3a3389
Commit
9e3a3389
authored
5 years ago
by
Nicolas Pope
Browse files
Options
Downloads
Patches
Plain Diff
More links in readmes
parent
c69e845e
No related branches found
No related tags found
No related merge requests found
Pipeline
#12958
passed
5 years ago
Stage: all
Changes
3
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
README.md
+1
-1
1 addition, 1 deletion
README.md
components/net/README.md
+10
-9
10 additions, 9 deletions
components/net/README.md
components/rgbd-sources/README.md
+27
-22
27 additions, 22 deletions
components/rgbd-sources/README.md
with
38 additions
and
32 deletions
README.md
+
1
−
1
View file @
9e3a3389
...
@@ -12,7 +12,7 @@ This monorepo contains all elements of the FTL software system.
...
@@ -12,7 +12,7 @@ This monorepo contains all elements of the FTL software system.
*
[
ftl-vision
](
applications/vision/
)
: Stereo vision node in p2p network, generates an RGB-Depth net stream
*
[
ftl-vision
](
applications/vision/
)
: Stereo vision node in p2p network, generates an RGB-Depth net stream
*
[
ftl-reconstruct
](
applications/reconstruct/
)
: Performs scene reconstruction from synchronised RGB-Depth sources
*
[
ftl-reconstruct
](
applications/reconstruct/
)
: Performs scene reconstruction from synchronised RGB-Depth sources
*
calibration-multi : All camera intrinsic and extrinsic calibration in one process
*
calibration-multi : All camera intrinsic and extrinsic calibration in one process
*
groupview : A quick camera viewing app that supports frame and video capture
*
[
ftl-view
](
applications/
groupview
/
)
: A quick camera viewing app that supports frame and video capture
*
[
ftl-gui
](
applications/gui/
)
: Desktop GUI
*
[
ftl-gui
](
applications/gui/
)
: Desktop GUI
*
front-end : Client side FTL code, both web and native
*
front-end : Client side FTL code, both web and native
*
web-service : A web backend service provider acting as a form of proxy
*
web-service : A web backend service provider acting as a form of proxy
...
...
This diff is collapsed.
Click to expand it.
components/net/README.md
+
10
−
9
View file @
9e3a3389
...
@@ -8,19 +8,20 @@ of socket connections but with a high bandwidth and low latency requirement
...
@@ -8,19 +8,20 @@ of socket connections but with a high bandwidth and low latency requirement
in sending and receiving on those sockets. Further work would be needed to
in sending and receiving on those sockets. Further work would be needed to
be efficient with large or massive numbers of sockets.
be efficient with large or massive numbers of sockets.
The protocol is based on top of MsgPack which works in both C++ and JavaScript.
The protocol is based on top of
[
MsgPack
](
https://github.com/msgpack/msgpack-c
)
To work with JavaScript the protocol works over TCP and TCP+Websockets. The
which works in both C++ and JavaScript. To work with JavaScript the protocol
library is also cross platform, supporting Windows and Linux.
works over TCP and TCP+Websockets. The library is also cross platform,
supporting Windows and Linux.
It is a template library, allowing simple RPC calls and bindings using the
It is a template library, allowing simple RPC calls and bindings using the
latest in C++ features such as optionals, lambdas and futures.
latest in C++ features such as optionals, lambdas and futures.
## Universe
## Universe
A Universe class
represents a network group and is the primary means of
A
[
Universe class
](
cpp/include/ftl/net/universe.hpp
)
represents a network group
interaction for the user of the library. It supports
`bind`
,
`connect`
,
`call`
,
and is the primary means of
interaction for the user of the library. It supports
`send`
,
`disconnect`
,
`asyncCall`
and more.
`bind`
,
`connect`
,
`call`
,
`send`
,
`disconnect`
,
`asyncCall`
and more.
## Peer
## Peer
A
fairly internal object that wraps a socket connection and deals with all
A
[
Peer object
](
cpp/include/ftl/net/peer.hpp
)
is a fairly internal object that
actual sending and receiving over
the network.
wraps a socket connection and deals with all
actual sending and receiving over
the network.
This diff is collapsed.
Click to expand it.
components/rgbd-sources/README.md
+
27
−
22
View file @
9e3a3389
# RGB-Depth Sources
# RGB-Depth Sources
This component provides a variety of sources for colour and depth images. These
This component provides a variety of sources for colour and depth images. These
include the following:
include the following
, but do not include virtual cameras
:
*
Intel Realsense depth camera
*
[
Intel Realsense depth camera
](
src/realsense_source.hpp
)
*
Stereo video from two video capture cards using a disparity algorithm
*
[
Stereo video
](
src/stereovideo.hpp
)
from two video capture cards using a disparity algorithm
*
Snapshots that were previously captured and saved to disk
*
[
Snapshots
](
src/snapshot_source.hpp
)
that were previously captured and saved to disk
*
Middlebury test datasets as available online
*
[
Middlebury
](
src/middlebury_source.hpp
)
test datasets as available online
*
Streamed network sources from other nodes
*
[
Streamed network sources
](
include/ftl/rgbd/streamer.hpp
)
from other nodes
An RGB-D source is represented as a two image channel object that is generated
An RGB-D source is represented as a two image channel object that is generated
through a pipeline of processes usually consisting of the following (but not
through a pipeline of processes usually consisting of the following (but not
...
@@ -18,29 +18,34 @@ all sources have all steps):
...
@@ -18,29 +18,34 @@ all sources have all steps):
## Groups
## Groups
A collection of sources may form a group that must be synchronised accurately
A collection of sources may form a group that must be synchronised accurately
for reconstruction to take place. A group class coordinates the above 4 steps
for reconstruction to take place. A
[
group class
](
include/ftl/rgbd/group.hpp
)
across all sources such that millisecond accurate frames with timestamps can
coordinates the above 4 steps across all sources such that millisecond accurate
be buffered and collected together to be passed on to the next stage. A high
frames with timestamps can be buffered and collected together to be passed on to
precision timer is used to manage the pipeline.
the next stage. A
[
high precision timer
](
../common/cpp/include/ftl/timer.hpp
)
is used to manage the pipeline.
## Streaming
## Streaming
One possible use for a group of sources is to stream them over a network
One possible use for a group of sources is to stream them over a network
where they may be re-grouped. A streamer object will receive sets of frames
where they may be re-grouped. A
[
streamer
](
include/ftl/rgbd/streamer.hpp
)
object
from a group object and then divide each image into a number of chunks, each
will receive sets of frames from a group object and then divide each image into
of which is compressed on a CPU core and sent to every client who asks for them.
a number of chunks, each of which is compressed on a CPU core and sent to every
Each client may ask for a different bitrate and resolution so the streamer will
client who asks for them. Each client may ask for a different bitrate and
also take care of this.
resolution so the streamer will also take care of this. The streamer class uses
the
[
ftl net library
](
../net/
)
for network communication.
## Calibration
## Calibration
Some sources require a camera calibration step. Lens corrections an stereo
Some sources require a camera calibration step. Lens corrections an stereo
camera configurations are applied by the calibrate class. Only stereo video
camera configurations are applied by the
[
calibrate class
](
src/calibrate.hpp
)
.
sources currently need this step. There is also some basic colour correction
Only stereo video sources currently need this step and the correction matrices
that can be applied.
are calculated using a separate
[
calibration app
](
../../application/calibration-multi/
)
. There is also some
basic
[
colour correction
](
src/colour.hpp
)
that can be applied.
## Disparity Algorithms
## Disparity Algorithms
A few algorithms are included with the RGB-D sources for converting two
A few algorithms are included with the RGB-D sources for converting two
colour images into one colour and one depth image based upon the pixel shift
colour images into one colour and one depth image based upon the pixel shift
observed between the two images. LibSGM is our algorithm of choice currently.
observed between the two images.
[
LibSGM
](
https://github.com/fixstars/libSGM
)
Further pre and post filtering and smoothing steps are applied, in particular
is our algorithm of choice currently. Further pre and post filtering and
an optical flow based temporal smoothing across a number of frames to reduce
smoothing steps are applied, in particular an optical flow based temporal
flickering effects.
smoothing across a number of frames to reduce flickering effects. This uses
\ No newline at end of file
[
NVIDIA's Optical Flow SDK
](
https://developer.nvidia.com/opticalflow-sdk
)
.
\ No newline at end of file
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment