Skip to content
Snippets Groups Projects
Commit 7c045a3e authored by Jari-Matti Mäkelä's avatar Jari-Matti Mäkelä
Browse files

Switch doc to Pages based CMS

parent 5ce451d7
Branches
No related tags found
No related merge requests found
...@@ -6,62 +6,6 @@ Merge requests are welcome - if you find an issue, [please report to us](issues/ ...@@ -6,62 +6,6 @@ Merge requests are welcome - if you find an issue, [please report to us](issues/
Compatible with: ShareLaTeX, pdfLaTeX, LuaLaTeX, XeLaTeX, LyX, latexmk. Compatible with: ShareLaTeX, pdfLaTeX, LuaLaTeX, XeLaTeX, LyX, latexmk.
<div align=center> ## Documentation
![](web/thesis.png) **<https://ttweb.utugit.fi/thesis/doc/overview/>**
</div>
## Main menu
* [Docker build environment](web/docker.md): read this if you want to generate
the document locally using Docker or use a Git service (Gitlab / Github)
for CI/CD.
* [LaTeX virtual machine](web/vm.md): read this if you want to write the
document locally using the preconfigured virtual machine (VM) or
USB distribution provided by the Department of Future Technologies.
The distribution provides: Debian stable, TeX Live, LyX, Pandoc,
TeX editors, Emacs, Vim, Pygments, Seafile, Chromium, Docker.
* [Using UTU ShareLaTeX](web/sharelatex.md): read this if you want to write
the document in the cloud using the real-time collaborative ShareLaTeX
service hosted by University of Turku.
* [Using the thesis template](web/config.md): **you should read this**.
Contains information on using and setting up the template.
* [PDF/A configuration](web/pdfa.md): **you should read this**.
Contains information on configuring the PDF/A settings.
## Extra features
- Should a required package be missing, see http://www.ctan.org/
- Consultation of the manuals of these packages is strongly encouraged
### R and LaTeX
- https://www.tidyverse.org/
- https://ggplot2.tidyverse.org/
- https://yihui.name/knitr/
- https://joshldavis.com/2014/04/12/beginners-tutorial-for-knitr/
- https://gitlab.utu.fi/jmjmak/knitr-examples
### Tools for benchmarking
- https://github.com/sharkdp/hyperfine
- https://github.com/Gabriel439/bench
### Including source code
- https://ctan.org/pkg/minted (preferred)
- https://ctan.org/pkg/listingsutf8 (does not support full unicode)
- https://ctan.org/pkg/sverb (still relevant?)
## Miscellaneous
- Report bugs via https://gitlab.utu.fi/ttweb/thesis/issues
- LaTeX tutorial: http://www.ctan.org/tex-archive/info/lshort/english/lshort.pdf
- Overleaf tutorials: https://www.overleaf.com/learn/latex/Tutorials
# Using the thesis template
## Structure
You're free to organize the project as you wish. A sane default structure
looks like this:
- root
- `.gitlab-ci.yml`: configures CI/CD
- `.gitignore`: configures git to ignore temp files
- `utuftthesis.clss`: document class description
- `thesis.xmpdata`: metadata for PDF/A generation
- `latexmkrc`: build instructions for latexmk
- `thesis.tex`: main document (links to sub-documents)
- `bibliography.bib`: biblioraphy data (multiple bibs also possible)
- `section-X.tex`: sub-documents for sections
- `images/`: embedded images
- `sources/`: embedded source code listings
In case you're using LyX, the following extra files are needed (the tex
files can be generated from the lyx files):
- root
- `utuftthesis.layout`: document class configuration for LyX
- `thesis.lyx`: main document (links to sub-documents)
- `section-X.lyx`: sub-documents for sections
The standard files included in the template consist of:
- The `latex` directory contains the document root. Everything from
this directory can be moved to the project root, but make sure you'll
update the GitLab CI, too.
- The `tests` directory can be deleted. It contains an extensive test
suite for the template.
- The `web` directory can be deleted. It contains this documentation.
- The document class file `utuftthesis.cls` is always required.
- The `*.lyx` and `*.layout` files are only needed if you decide to
use [LyX](https://www.lyx.org/), and can be deleted in other cases.
## Workflow
The documents are typically edited with
- TeX editors
- generic editors
- LyX
- hosted cloud services ([ShareLaTeX, Overleaf](sharelatex.md), or even GitLab Web IDE)
The documents are compiled with
- TeX → directly with TeX utilities (pdflatex/xelatex/tectonic) → PDF
- TeX → LaTeX frontend (mklatex, arara etc.) → PDF
- LyX → TeX → one of the previous two → PDF
- LyX → PDF
- [Integrated build system (ShareLaTeX / Overleaf) → PDF](sharelatex.md)
- [CI/CD pipeline (GitLab) → PDF artifact / publish site](docker.md#setting-up-gitlab-cicd)
Note that when using TeX utilities directly, you need to make sure the
document has been fully processed. Usually this requires multiple
compilation passes, interleaved with invocations of the auxiliary tools
(glossary & bibliography generation etc.). Using the LaTeX frontend
tools (latexmk) instead are highly recommended.
## Document class configuration
The thesis document class is described in [`utuftthesis.cls`](latex/utuftthesis.cls).
- Based on the standard LaTeX `report` class
- Proper page sizes as required by university guide for students:
- proper font sizes as well as linespacings
- proper size of margins
- Document parameters (command: `\documentclass[the,parameters]{utuftthesis}`)
- `version`: `draft` / `final` (default: `draft`) shows/hides [draft] in the footer
- `language`: `finnish` / `english` (default: `finnish`) defines the main document
language, affects the general document appearance and hyphenation
- `mainfont`: (default: `times`)
- pdftex: main font package (`lmodern`, `fourier`, ...), special value: `times`, `none`
- xetex: main font name (Arial, Comic Sans, ...), special value: `times` enables Times New Roman, `none`
- note: use `none` in cloud/CI services in place of `times` unless you provide the
fonts yourself. The copyrighted fonts cannot be included in free platform
distributions due to licensing issues.
- `hidechapters`: `true` / `false` (default: `true`) hides/shows the chapter/luku
text at the beginning of each Chapter
- `countbibpages`: `true` / `false` (default: `false`) include bibliography
pages when calculating the total number of pages
- `sharelatex`: `true` / `false` (default: `false`) don't attempt to locate
copyright protected system fonts, instead read them from the project
repository
- `emptyfirstpages`: `true` / `false` (default: `true`) clear the headers/footers
for the 1st pages of text chapters
- Extra document parameters (command: `\documentclass[extra,parameters]{utuftthesis}`)
- `paper`: (default: `a4paper`) paper size
- `pagecountdelta`: any integer (default: `-1`). Adjust the counting of app. pages
- `pdfaconformance`: `a-1b`/ `a-2b` / `a-2u` / `a-3b` / `a-3u` / `none`
(default: `a-2b`) PDF/A conformance level. A special parameter `mathxmp` can be
also appended (see the PDF/A documentation).
- `hidelinks`: `true` / `false` (default: `true`) hide the underlining of links
- `minted`: `true` / `false` (default: `false`) use minted instead of listings
for embedded code listings. Note that minted requires the use of `-shell-escape`
build parameter.
- `biburlbreak`: `ragged` / `split` / `none` (default: `split`) how to handle
long urls in the bibliography
- Publication metadata
- `\pubyear{2000}`: the year of the publication
- `\pubmonth{6}`: the month of the publication
- `\publab{labname}`: the name of the laboratory, e.g. tietojenkäsittely
- `\publaben{labname}`: the name of the laboratory in english, e.g. computer science
- `\pubtype{type}`: the type of the thesis, special type name (`tkk`,
`luk`, `gradu`, `di`), or a custom string for both fi/en
- `\title{titlename}`: the title of the publication
- `\author{authorname}`: the author of the publication
- Abstract
- `\abstract{text}`: abstract written in the main document language
- `\abstracten{text}`: abstract written in english
- `\keywords{kw1,kw2}`: list of keywords in the main document language
- `\keywordsen{kw1,kw2}`: list of keywords in english
- Title page
- `\maketitle`: the standard LaTeX command for creating a title page. This template
also creates the pages for abstracts
- TOC and lists
- `\tableofcontents`: generates the table of contents
- `\listoffigures`: generates the list of figures
- `\listoftables`: generates the list of tables
- `\listofacronyms`: generates the list of acronyms (using the `nomencl` package)
- Main content
- `\begin{document}`: begins the main content
- `\end{document}`: ends the main content
- `\chapter{name}`: generates a new chapter (1)
- `\section{name}`: generates a new section (1.1)
- `\subsection{name}`: generates a new section (1.1.1)
- `\printbibliography`: generates the bibliography list
(required even if no bibliography has been used!)
- `\appchapter{name}`: generates a new appendix chapter (A) after the bibliography list
- Entry environment:
- the actual items are aligned to suit the widest label, which is
given as an argument to the environment
```
\begin{entry}[widest label]
\item[1st label text] ...
\item[2nd label text] ...
\end{entry}
```
## PDF/A conformance and metadata
The PDF/A generation mechanism is described in more detail in the
[pdfx manual](ftp://ftp.funet.fi/pub/TeX/CTAN/macros/latex/contrib/pdfx/pdfx.pdf).
### Configuration
- First, compile the document using `version=final` in the document class parameters
- The PDF/A conformance level can be configured with the `pdfaconformance` parameter
- The default conformance level is `a-2b` (*PDF/A-2b: PDF/A-2b – Level B (basic) conformance*)
- other supported values: `a-1b`, `a-2u`, `a-3b`, `a-3u`, `a-1a`, `a-2a`, `a-3a`, and `none`
- the values `a-1a`, `a-2a`, and `a-3a` are still marked as experimental
- the default conformance level should be sufficient for most thesis documents
- `none` disables pdfx
### Metadata file
- The pdfx package explicitly disables the propagation of document metadata
from the LaTeX preamble to PDF metadata fields (such aspdfauthor, pdftitle,
pdfsubject, pdfkeywords, etc.). This should not be altered with hyperref
as all the embedded metadata must be synchronized in conformant PDF/A
documents.
- Instead, the embedded PDF/A metadata is provided in XMP format and read from
`<maindoc>.xmpdata` (just replace `<maindoc>` here with the main tex file,
e.g. `thesis`).
- For detailed documentation of the metadata fields and the process, please
refer to the pdfx.pdf sections 2.2--2.3.
- The following code ([`thesis.xmpdata`](latex/thesis.xmpdata)) is provided as an example:
```
\Title{Baking through the ages}
\Author{A. Baker\sep C. Kneader}
\Language{en-GB}
\Keywords{cookies\sep muffins\sep cakes}
\Publisher{University of Turku}
```
## Using the template
### Git
Start by cloning the whole template repository (or fork the project
in GitLab).
The following command line instructions can be used for setting up
a new thesis (replace MYNAME with your utu username):
```bash
$ git clone https://gitlab.utu.fi/ttweb/thesis
$ cd thesis
$ git remote add mythesis https://gitlab.utu.fi/MYNAME/mythesis
$ git push -u mythesis master
```
If you'd like to use the GitLab CI script, considering setting up
the simplified CI pipeline that does not run the full test suite
(invoke these commands in the thesis/ directory):
```bash
$ cp .gitlab-ci-simple.yml .gitlab-ci.yml
$ rm -fr tests
$ git add -A
$ git ci -m 'Removed unnecessary testing.'
$ git push
```
Changes to the project will be pushed to your `mythesis` repository
(invoke these commands in the thesis/ directory):
```bash
$ touch test
$ git add test
$ git ci -m 'Just testing'
$ git remote -v
$ git push
```
### Latexmk (PdfTeX)
PdfTeX is the default engine for compiling the template. It offers
better compatibility with most features.
When using latexmk, the project can be compiled like so:
```bash
$ latexmk -pdf -shell-escape thesis.tex
```
### Latexmk (XeTeX)
XeTeX / XeLaTeX version is required if you want to use
OpenType/TrueType fonts.
When using latexmk, XeLaTeX version can be compiled like so:
```bash
$ latexmk -xelatex -shell-escape thesis.tex
```
### LyX
LyX provides a graphical WYSIWYM interface for editing TeX-like
LyX documents that can be converted to TeX. The template distribution
contains the `utuftthesis.layout` document class description file
for the same `utuftthesis.cls` as the one used by TeX documents.
Start by editing the provided `thesis.lyx`. The file has been configured
to use the custom `utuftthesis` document class.
Depending on your LyX version, you may need to enable `-shell-escape`
from the global LyX settings. Follow the program's instructions.
When updating your TeX distribution, LyX may require reconfiguration.
This functionality is available via the tools menu.
# Docker build environment
A custom LaTeX build environment based on *Debian stable* has been developed
using *Docker* based container technologies. The purpose of this container
is to facilitate LaTeX document generation using GitLab CI/CD pipelines. The
same image can also be used for generating documents either using the standard
command line prompt or as a backend for new middleware.
The container's TeX distribution is based on
[TinyTeX](https://yihui.org/tinytex/) + TeX 'small' scheme + additional
packages. See the [`Dockerfile`](Dockerfile) for a full list of installed
TeX packages. In addition, a headless Java JRE is provided for
[VeraPDF](https://verapdf.org/) powered [PDF/A validation](pdfa.md), poppler utils
for analyzing the generated PDF files (`pdfinfo`, `pdffonts` etc.), and
python3 + pygments for pretty printing source code using the LaTeX
[minted](https://ctan.org/pkg/minted) package.
Docker image available at Docker Hub: https://hub.docker.com/repository/docker/konttipoju/tinytex
## Setting up GitLab CI/CD
The project's [GitLab CI script](.gitlab-ci.yml) provides an rather complex
example of using the Docker image in GitLab in various ways (building documents,
asserting certain properties, validating PDF/A conformance).
[This simplified version](.gitlab-ci-simple.yml) can be used instead:
```yaml
# see https://hub.docker.com/repository/docker/konttipoju/tinytex
image: konttipoju/tinytex:2020-04a
# build latex/thesis.tex -> latex/thesis.pdf using pdflatex
# shell-escape functionality is required by the minted package
build:
stage: build
script:
- cd latex
- latexmk -pdf --shell-escape thesis.tex
artifacts:
paths:
- latex/thesis.pdf
# a validation prints "PASS + something" is the file is PDF/A conformant
validate:
stage: test
script:
- pdfa-validate latex/thesis.pdf|grep '^PASS'
```
You're also free to consider other LaTeX Docker images such as these:
* https://github.com/blang/latex-docker
* https://github.com/natlownes/docker-latex
## Installing new TeX packages
If a certain package is missing, the Dockerfile can be modified, but adding few
extra packages is even simpler by just invoking `tlmgr`:
```bash
$ tlmgr install packagename
```
The packages can be installed even inside the CI script. However, in case the
document will be recompiled multiple times, generating a new Docker image and
caching it locally is recommended. Otherwise the public repositories might
block access to fight the bandwidth costs.
### Validating PDF/A conformance
A headless Java 11 JRE is provided in the Docker image for VeraPDF powered
PDF/A validation:
```bash
$ pdfa-validate thesis.pdf
```
See the [PDF/A page](web/pdfa.md) for further instructions.
## Compiling the thesis using Docker
We assume the thesis is located in the local folder `thesis/`. The currently
active user also needs to belong in the 'docker' group. Otherwise try
`sudo docker` in place of plain `docker`. In addition, the Docker
service needs to be active (try `systemctl start docker`).
The build environment first needs to be fetched from Docker Hub:
```bash
$ docker pull konttipoju/tinytex:2020-04a
```
We can then list all the available container images:
```bash
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
konttipoju/tinytex 2020-04a 4e7da92afcfa 2 hours ago 765MB
debian buster-slim 2dbffcb4f093 4 weeks ago 69.2MB
```
An interactive shell can be launched in the LaTeX container. This shell
provides access to compile the thesis:
```
$ docker container run -it --mount type=bind,source=$(pwd)/thesis,target=/thesis 4e7da92afcfa /bin/bash
root@d662b10721e7:/var/local# cd /thesis
root@d662b10721e7:/tex# latexmk -pdf --shell-escape thesis.tex
```
The results of the compilation will be available in the `thesis/` directory.
In case the compilation succeeds and the document was compiled in the `final`,
not `draft` mode, the resulting `thesis.pdf` should be a PDF/A conformant
final version.
web/overleaf.png

98.1 KiB

# PDF/A conformance and metadata
The PDF/A generation mechanism is described in more detail in the
[pdfx manual](ftp://ftp.funet.fi/pub/TeX/CTAN/macros/latex/contrib/pdfx/pdfx.pdf).
## Configuration
- First, compile the document using `version=final` in the document class parameters
- The PDF/A conformance level can be configured with the `pdfaconformance` parameter
- The default conformance level is `a-2b` (*PDF/A-2b: PDF/A-2b – Level B (basic) conformance*)
- other supported values: `a-1b`, `a-2u`, `a-3b`, `a-3u`, `a-1a`, `a-2a`, `a-3a`, and `none`
- the values `a-1a`, `a-2a`, and `a-3a` are still marked as experimental
- the default conformance level should be sufficient for most thesis documents
- `none` disables pdfx
## Metadata file (thesis.xmpdata)
- The pdfx package explicitly disables the propagation of document metadata
from the LaTeX preamble to PDF metadata fields (such aspdfauthor, pdftitle,
pdfsubject, pdfkeywords, etc.). This should not be altered with hyperref
as all the embedded metadata must be synchronized in conformant PDF/A
documents.
- Instead, the embedded PDF/A metadata is provided in XMP format and read from
`<maindoc>.xmpdata` (just replace `<maindoc>` here with the main tex file,
e.g. `thesis`).
- For detailed documentation of the metadata fields and the process, please
refer to the pdfx.pdf sections 2.2--2.3.
- The following code (`thesis.xmpdata`) is provided as an example:
```
\Title{Baking through the ages}
\Author{A. Baker\sep C. Kneader}
\Language{en-GB}
\Keywords{cookies\sep muffins\sep cakes}
\Publisher{University of Turku}
```
## Tools
Note that PDF/A support is a rather new feature in LaTeX, e.g. the pdfx
1.6.3 library was announced 2019/2/27. If the document fails to validate,
consider compiling with a more recent LaTeX distribution. The template
has been tested with TeXLive 2017+.
### VeraPDF (generic instructions)
A free [VeraPDF](https://verapdf.org/) tool can be used to test if the
resulting PDF files are PDF/A conformant. Unfortunately, the official
VeraPDF distribution may not build. Use the snapshot
[version hosted at UTU gitlab](https://gitlab.utu.fi/jmjmak/veraPDF-apps)
instead.
The following script shows how to invoke the VeraPDF tool:
```bash
$ wget https://gitlab.utu.fi/jmjmak/veraPDF-apps/raw/integration/greenfield-apps-1.15.0-SNAPSHOT.jar?inline=false -O validator.jar
$ latexmk -pdf -shell-escape thesis.tex
$ java -cp validator.jar org.verapdf.apps.GreenfieldCliWrapper --format text -v thesis.pdf
```
### VeraPDF (Docker image)
A headless Java 11 JRE is provided in the Docker image for VeraPDF
powered PDF/A validation. The UTU build of the VeraPDF tool is also
included in the [LaTeX Docker image](web/docker.md) and can be run by
simply invoking:
```bash
$ pdfa-validate thesis.pdf
```
The [GitLab CI script](.gitlab-ci-simple.yml)
in this project demonstrates the validation of CI/CD generated PDFs with
this tool.
## Limitations
- pdfTeX should not have any significant limitations
- luaLaTeX expects all input documents to be encoded using utf-8 (mainly
a problem for Windows users)
- XeLaTeX generated documents may fail when using OTF fonts (see the
[pdfx manual](ftp://ftp.funet.fi/pub/TeX/CTAN/macros/latex/contrib/pdfx/pdfx.pdf)
section 3.1.1).
- XeLaTeX requires the extra parameter (`-output-driver="xdvipdfmx -z 0"`)
which is included in the provided `latexmkrc`.
- XeLaTeX will not generate conforming PDF/A documents when the
output-directory is changed in `latexmk`.
- Configuring a default action (hyperref) when opening the PDF will
produce non-conforming PDF files.
## What are these PDF/A conformance rules?
The VeraPDF page contains a list of validation rules:
* https://github.com/veraPDF/veraPDF-validation-profiles/wiki/PDFA-Part-1-rules
* https://github.com/veraPDF/veraPDF-validation-profiles/wiki/PDFA-Parts-2-and-3-rules
Related:
* https://utuguides.fi/opinnayte/pdfa
* https://coursepages.uta.fi/mtta1-latex/pdf-a/
# Setting up ShareLaTeX
A ShareLaTeX platform is hosted on-premises at UTU Software Engineering Laboratory: https://tex.soft.utu.fi/
Press register at the top right corner to register for the service.
See the general LaTeX document guide for configuring the template.
The following set of screenshots describes the process of exporting the
thesis template from the GitLab to ShareLaTeX (exporting to Overleaf
differs slightly, but this instruction can be applied):
## Step 1: export the project
<div align=center>
![](sharelatex1.png)
</div>
## Step 2: store the zip file locally
<div align=center>
![](sharelatex2.png)
</div>
## Step 3: create a new ShareLaTeX project
<div align=center>
![](sharelatex3.png)
</div>
## Step 4: use the zip (step 2) as project template
<div align=center>
![](sharelatex4.png)
![](sharelatex5.png)
![](sharelatex6.png)
</div>
## Step 5: delete the tests folder
<div align=center>
![](sharelatex7.png)
</div>
## Step 6: set up the main TeX document
<div align=center>
![](sharelatex8.png)
![](sharelatex9.png)
</div>
## Step 7: recompile using the new main document
<div align=center>
![](sharelatex10.png)
</div>
## Done
web/sharelatex.png

22.9 KiB

web/sharelatex1.png

33.8 KiB

web/sharelatex10.png

77.2 KiB

web/sharelatex2.png

8.49 KiB

web/sharelatex3.png

31.9 KiB

web/sharelatex4.png

8.84 KiB

web/sharelatex5.png

12.3 KiB

web/sharelatex6.png

9.75 KiB

web/sharelatex7.png

69.6 KiB

web/sharelatex8.png

66.7 KiB

web/sharelatex9.png

84.5 KiB

web/thesis.png

6.16 KiB

# LaTeX virtual machine
The Department of Future Technologies provides a set of virtual machines for
software development and other tasks via https://ftdev.utu.fi/. The `utuvm-latex`
virtual machine is especially preconfigured for generating LaTeX documents.
The virtual machine has been tested to build this document template without
any issues.
The `utuvm-latex` virtual machine contains preinstalled versions of
* TeX Live (popular Linux/Unix LaTeX distribution)
* LyX (for WYSIWYM editing)
* Pandoc for generating LaTeX documents from markdown sources
* TeX editors (Gummy, Texworks, Texmaker, Texstudio)
* Generic editors (Emacs, Vim, Geany, nano)
* PDF reader (Evince)
* Pygments for syntax highlighting (minted)
* Seafile client for storing the documents in the University's
Seafile server.
* [Docker](docker.md#compiling-the-thesis-using-docker), systemd-nspawn
The virtual machine also comes with a modern browser (Chromium) for editing
documents in ShareLaTeX and Overleaf.
Extra applications can be easily installed from deb packages.
![](web/vm.png)
## Using the virtual machine
Please refer to the instructions on the virtual machine site for
setting up the platform.
The TeX tools in the VM are standard without further customizations.
The [general instructions on using the template](web/config.md)
should cover the usage of the provided tools.
web/vm.png

132 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment