.. _Tomoh52nx:

h52nx tutorial
==============

.. article-info::
    :read-time: 10 min read
    :class-container: sd-p-2 sd-outline-muted sd-rounded-1

the `h52nx` application can be used to convert acquisition from hdf5/bliss to a nexus - NXtomo compliant format.
This format will also be stored in .h5 / .hdf5 / .nx file.
For comprehension we will use the nexus format (.nx) in this tutorial.

To call this application you can call directly

.. code-block:: bash

   $ nxtomomill h52nx [options]


.. tip::

   You can also call it direcly from python executable using:

   .. code-block:: bash

      $ python -m nxtomomill h52nx [options]


Then you can convert bliss h5 file :ref:`h52nx_without_config_file` or :ref:`h52nx_with_config_file`


.. _h52nx_without_config_file:

without configuration file
--------------------------

The first two parameters should be:

* input-file-name: bliss.hdf5 master file, containing the details of the acquisition
* output-file: destination file where result will be store.

Sor for example to convert a file name 'bliss.hdf5' to a 'my_nxtomo_file.nx' you should call:

.. code-block:: console

   $ python -m nxtomomill h52nx bliss.hdf5 my_nxtomo_file.nx

.. tip::

   you can provide an output directory instead of a file path. Then the file basename will remane the same, only the extension will be changed to .nx

   .. code-block:: console

      $ python -m nxtomomill h52nx bliss.hdf5 .

.. versionchanged:: v0.13

   if the output is not provided the default behavior is to create the NXtomo to the PROCESSED_DATA directory. To overstep this you can provide a directory.

You can also access the help of h52nx by calling:


.. code-block:: console

   $ python -m nxtomomill h52nx --help


By default if some information are missing the converter will ask you for missing input (it can be the case of the incoming energy for example).
If you want to avoid converter for information you can add the `--no-input` option.

An acquisition file can contain several sequence (so several acquisition).
`h52nx` will convert them all create one file per acquisition and one file referencing all acquisitions.
You can ask the converter to keep all the acquisition into a single file using the '--single-file' option.


Input type
^^^^^^^^^^

z-series
""""""""

z-series are handled by nxtomomill since 0.4. It will create one entry per 'z' found.

h52nx-settings
^^^^^^^^^^^^^^

The `h5tonx` command is using the 'h5_to_nx' function from 'converter' module.
This will used by default :ref:`h52nx-settings`  defined in nxtomomill.settings.py file
If you want you can overwrite them from the command line.

camera name
"""""""""""

In order to know if an NXDetector should be converted or not it relies on 'H5_VALID_CAMERA_NAMES' defined in nxtomomill.settings.py.
 * If the value is None (default) then we will first try to retrieve group (under instrument) that are defined as an 'NXdetector' NX_class. If none are found then we try to retrieve group that 'looks' like a detector (containing a dataset name 'data' and which is of dimension 3).
 * It can also be set as a tuple of string. Each string is a detector name to be handle. Those can handle Linux wildcard (like 'frelon*')
 * the values defined in settings can be overwrite by `--valid_camera_names` option (see help).


other (rotation angle, translation...)
""""""""""""""""""""""""""""""""""""""

Most of the key used to retrieve other operation can be overwrite from
an option from command line

.. program-output:: nxtomomill h52nx --help

.. _h52nx_with_config_file:

with a configuration file
-------------------------

Generate a default configuration file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can create a default configuration file by calling

.. code-block:: bash

    $ nxtomomill h5-config [output_conf_file.cfg]

This will generate a configuration file based on the existing :ref:`Settings` that you can edit.

Once the configuration file fit your needs you can execute it by calling h52nx.

.. code-block:: bash

    $ nxtomomill h52nx input_bliss_file.h5 [output_nexus_file.nx] --config [output_conf_file.cfg]

.. warning::

    when you provide a configuration file to h52nx then no other input can be provided (excepted for the output file which is optional).

.. note:: If you want to provide configuration from scan titles you can ignore the `FRAME_TYPE_SECTION` and generate a default configuration file with the option `--from-title-names`.
          On the contrary if you want to ignore the `ENTRIES_AND_TITLES_SECTION` you can use the `--from-scan-urls` option.

Example of a configuration file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. include:: resources/default_hdf5_config.cfg
   :literal:


.. warning:: the "data_scans" values from the `FRAME_TYPE_SECTION` section should be provided with indentation (at least one empty space. Otherwise they will be ignored by configparser.


.. versionadded:: v0.9

   There is a dedicated section regarding PCO tomo.
   If the bliss acquisition is a detected as a multi-tomo by default it will generate `nb_tomo` * `nb_loop` NXtomo.
   And those will start from the first projections found.

   It might append you want to 'refine' NXtomo to generate and this is the goal of this section.

   Here is the details of the parameters:

   * `angle offset` (the *projections before will be ignored*. This will not affect the dark and the flat that will always be copied for pco tomo)
   * `tomo_n`: how many NXtomo you want to generate
   * `angle_interval_in_degree`: do you want to create NXtomo which will cover 180 or 360 degrees.
   * `shift_angles`: do you want to reset angles of the final NXtomo to 0

.. versionadded:: v2.1

   Keys used in the 'KeysSection' like `valid_camera_names`, `rotation_angle_keys`, `sample_x_keys`... can be defined within a context like:

   .. code-block:: ini

      # Valid camera names.
      valid_camera_names = ('tomo_config:tomo_detector',)
      # Keys for sample x translation.
      sample_x_keys = ('id11:tomo_config:sample_x', 'samx', 'psx', 'sax', 'fake_sx')

   Here:

   * `valid_camera_names` first key is defining a `tomo_config` context.
   * `sample_x_keys` first key is defining a `tomo_config` and a beamline context.

   **Contexts**

   Contexts improve flexibly and defining rules on how and when to access data. Think of contexts as "hints" or "tags" that tell the system:

   - **where**: those are context like 'tomo_config'. It tells to directly refeer to some extra-metadata contained in bliss-tomo config instead of 'raw' dataset.
   - **when** - adapt to different setup: those are `beamline` context. This key will be used only if the dataset comes from this beamline.

   Example: id16a:bliss_tomo:sample_u means "look for sample_u in the bliss_tomo configuration, but only if the data comes from the id16a beamline."


   Contexts can be given as a prefix of key. Separator used is `:`. Today we have two types of context:

   * 'tomo_config' context. If given this mean that a mapping will be done with the value(s) contained with the named "tomo_config" dataset. If the file doesn't contains a bliss-tomo config this key will be ignored.
   * 'beamline specific' context. If a key is given with a bemline specific contains then this key will be used only if the acquisition comes from this specific beamline. Else this key will be ignored.

   .. note::

      The given configuration will be preprocessed to update and filter keys according to the file converted.

   .. warning::

      Contexts can be provided in any order, but must always precede the dataset name. The dataset name should always be the final part of the key.

Constrain to the conversion
---------------------------

*nxtomomill h52nx* will create an `HDF5 Virtual dataset <https://docs.h5py.org/en/stable/vds.html>`_ to create the /instrument/detector/data dataset.
This `HDF5 Virtual dataset <https://docs.h5py.org/en/stable/vds.html>`_ require the creation of a `VirtualLayout <https://docs.h5py.org/en/stable/vds.html#h5py.VirtualLayout>`_ which force all the frame to the of the same data type (uint16, int32...)

So if you have bliss scans (entries) with different data type the conversion will fail. If this happen then there is a bug wuith the bliss acquisition.

Nevertheless there is a workaround for it. You can add to your ``instrument/{detector_name}`` group a dataset named ``data_cast`` near the ``data`` dataset.
If it exist then it will be picked by *nxtomomill h52nx* instead of the ``data`` dataset. Of course this second dataset must have a coherent type with the rest of the acquisition.

.. warning::

   Be caution with the casting.

.. hint::

   You can find `here a script casting a dataset to another type <https://gitlab.esrf.fr/tomotools/scripts/-/blob/master/cast_detector_dataset.py>`_


.. hint::

   Consult section on :ref:`handling_h52nx_issues` in case of troubles.


Python API
----------

You can also convert to NXtomo directly from the Python API using the :class:`TomoHDF5Config` class and :func:`from_h5_to_nx` function like:

.. code-block:: python

   from nxtomomill.converter import from_h5_to_nx
   from nxtomomill.models.h52nx import H52nxModel

   config = H52nxModel(
      input_file=...,
      output_file=...,
   )
   from_h5_to_nx(
      configuration=config,
   )
