A few examples

This page will show you some examples to work with TTT images using an ICR.

Binning data

Let’s assume you have obtained a large amount of data. A first, natural need would be to bin this data and download it to your computer. Although we strongly recommend processing all of your data and running your custom algorithms inside the ICR, let’s see an example on how to bin images and download them locally.

First, we will assume your Python logic is contained in a file called cut-and-bin.py. You can find an example of such code below.

cut-and-bin.py

import os
import fnmatch
import sys
import numpy as np
from astropy.io import fits
from astropy.wcs import WCS
from astropy.nddata import Cutout2D, block_reduce
from astropy.wcs.utils import proj_plane_pixel_scales


def find_fits_files(origin_folder, name_filter):
    matches = []
    for root, _, filenames in os.walk(origin_folder):
        for filename in fnmatch.filter(filenames, f"*{name_filter}*.fits"):
            matches.append(os.path.join(root, filename))
    return matches


def cut_and_bin(filelist, result_path, crop_size_x, crop_size_y, binning):
    os.makedirs(result_path, exist_ok=True)
    
    for f in filelist:
        with fits.open(f) as hdu:
            try:
                data = hdu[0].data.astype(np.float32)
                header = hdu[0].header
            except Exception as e:
                print(f"{f} is not a valid HDUL: {e}")
                continue

        wcs = WCS(header)
        image_shape = data.shape
        center_x = image_shape[1] / 2
        center_y = image_shape[0] / 2

        cutout = Cutout2D(data, (center_x, center_y), (crop_size_y, crop_size_x), wcs=wcs)
        cropped_data = cutout.data
        cropped_wcs = cutout.wcs

        binned_data = block_reduce(cropped_data, block_size=(binning, binning), func=np.mean)

        new_header = header.copy()
        new_header.update(cutout.wcs.to_header())
        new_file = os.path.join(result_path, os.path.basename(f))
        fits.writeto(new_file, binned_data, new_header, overwrite=True)
        print(f"Saved cut and binned file to {new_file}")


def main():
    if len(sys.argv) != 7:
        print("Usage: python cut-and-bin.py [origin-folder] [name] [cut_x] [cut_y] [bin] [destination-folder]")
        sys.exit(1)

    origin_folder = sys.argv[1]
    name_filter = sys.argv[2]
    cut_x = int(sys.argv[3])
    cut_y = int(sys.argv[4])
    binning = int(sys.argv[5])
    destination_folder = sys.argv[6]

    fits_files = find_fits_files(origin_folder, name_filter)
    if not fits_files:
        print(f"No FITS files found in {origin_folder} matching name '{name_filter}'")
        sys.exit(1)

    cut_and_bin(fits_files, destination_folder, cut_x, cut_y, binning)


if __name__ == "__main__":
    main()

This Python script is run from a command-line like so:

python cut-and-bin.py [origin-folder] [search-strings] [cut_x] [cut_y] [bin] [destination-folder]

Search for all files with the strings [search-strings] (a comma-separated list) inside the folder [folder-name].
Crop all the found images to a size [size-x], [size-y]. The cropping is relative to the center of the image.
Bin the image using [bin] × [bin] dimensions and a median filter, in order to remove salt and pepper. [bin] should be an integer, divisor of both [size-x] and [size-y]
Save the output on the [output-folder], which should be an existing folder.

Copy the code above and create a new file in your ICR, pasting the code inside:

nano cut-and-bin.py

Create an output folder for your binned images:

mkdir output

You can now run this script. For example, if you were to bin images from the target Scheila, on the night of the 25th of March 2025, you would run:

python cut-and-bin.py /proposal/[UUID]/red "2025-03-25,Scheila" 10000 10000 4 output

This will first crop any images from Scheila on the 25th of March to 10 000 px × 10 000 px, then they will be binned down to 2500 × 2500, and they will be saved to the folder output.

If the script isn’t working properly, there are a few common issues to check:

It is possible that the executable names (python3) haven’t been added to the PATH.
It is possible that python packages like astropy or numpy haven’t been installed. Try pip install numpy, for instance.

Setting up a Jupyter Notebook server

It is possible to set up a Jupyter Notebook server (or any other service-like application) on the ICR and access it via SSH tunneling. For example:

Access to your ICR via SSH.
Make sure Python and Jupyter are installed:

sudo apt-get update
sudo apt-get install python-pip
pip install jupyterlab
pip install notebook

Start a Jupyter server on your ICR:

jupyter notebook

By default, the notebook will run on the port 8888. We can now set up an SSH tunnel with port forwarding in order to use our local computer to connect to the notebook on the ICR. Open a new command-line tool and run:

ssh -L 80:localhost:8888 -p 3336 [ICR-name]@icr.lightbridges.es

This is effectively tunneling your localhost:80 local address to port 8888 at your ICR. If you now open a web browser and type localhost:80, you should be able to see your Jupyter notebook up and running!