I kind of stumbled into setting up a DIY astro cam through several earlier articles, and learning some ins and outs of the telescopes and cameras along the way. By the end of those articles I wasn’t entirely pleased with the results I got so I felt the urge to dig deeper. I started my adventures by writing a simple bash script and using tools such as libcamera-still to capture the RAW files and ssh to copy over the pictures, but this was not performing good at all so this felt like an interesting thing to improve. So I started to explore some options here.
Pre-setup
Make sure that you have a Raspberry Pi with Raspbian OS setup. In my case it’s a RPI2 with Raspbian 12 (bookworm). Also make sure to have SSH access and the disc space expanded to the entire sd-card space. You should also hookup the camera to the RPI:
And install the correct device tree overlay for your camera. In my case I had to edit the boot config and set:
#Camera
dtoverlay=imx462,clock-frequency=74250000The libcamera library and userspace demo applications like libcamera-still, libcamera-vid and so on should already come pre-installed.
libcamera, libcamera-apps, rpicam-apps, pycamera2
Until now I’ve been testing with the utilities that come with Raspbian OS, being libcamera-still and friends. But what er all of these software packages exactly?
- libcamera: a modern C++ library that abstracts the usage of cameras and ISPs away to make application development easier and with less gory camera specifics. Libcamera is developed as an independent open-source project for linux and Android.
- libcamera-apps: a bunch of userspace applications that are build upon libcamera. It allows users to easily snap pictures, RAWs and videos using image sensors and ISPs that are supported through libcamera. It’s developed by the Raspberry Pi Foundation, therfor the libcamera library and libcamera-apps are developed by 2 different entities. More recently the apps/tools have been renamed to rpicam-apps to elaborate that the userspace apps and libcamera are 2 different things being supported by different teams.
- rpicam-apps: previously named libcamera-apps, as you could read in the above
- pycamera2: python library to make applications using libcamera as backend. It replaces the pycamera library that was created on top of the legacy Raspbian camera stack. As a python library it’s for many people a more convenient way to start hacking vision apps compared to directly using libcamera in a C++ project. Picamera2 also comes with a nice manual to get you going.
I started experimenting with pycamera2 myself a bit, but since I wanted a network solution I also started to think what other stuff I would need to develop. A REST based API? Maybe something with websocket for fast response? And how does that work in bad network conditions? Or could I maybe sail on the work of others? Well… meet INDI.
INDI
To quote their own words:
INDI Library is an open source software to control astronomical equipment. It is based on the Instrument Neutral Distributed Interface (INDI) protocol and acts as a bridge between software clients and hardware devices. Since it is network transparent, it enables you to communicate with your equipment transparently over any network without requiring any 3rd party software. It is simple enough to control a single backyard telescope, and powerful enough to control state of the art observatories across multiple locations.
INDI serves offers the networked approach that I’ve been achieving so by calling my libcamera commands over SSH, and it also has libcamera support so it fits my goal perfectly! But INDI is also a broad collection of many other software pieces coming together, not only for our Raspberry Pi based cameras but for many other cameras, controllers, motorized mounts and so forth. But let’s try to focus on some of the components that are of most interest for us.
indi, indi-libcamera, indi-pylibcamera, indi-3rdparty
- indi: the core library
- indi-3rdparty: a collection of all sorts of specific driver implementations for INDI.
- indi-libcamera: this is just the specific 3rd party INDI driver for devices that are supported by libcamera. It’s basically just one of the many drivers in indi-3rdparty.
- indi-pylibcamera: developed as an alternative driver implementation for indi-libcamera. However in contrary to the latter indi-pylibcamera is not part of the indi-3rdparty repository and probable will never be
I started by going through many pages of developer ramblings at the indib forum. From that Indi-pylibcamera seems to have matured best over the years and the author is very willing to help out with any issues that you have. But given it’s a alternative to the more official 3rd-party drivers repository I’m hesitating if it’s the best choice in the long run. The other way around the indi-libcamera driver doesn’t seem to be well maintained but I was willing to give a helping hand in case it was required. So let’s get started.
Compiling from INDI from source on Raspbian 12 (bookworm)
You can off course try to apt install all key components, but in my case I would end up with slightly outdated software and with libcamera support you mostly want the latest and greatest. Furthermore if I would be willing to help out with the development or debugging I’ll need to compile from source anyway. So let’s get our hands dirty…
To get the latest working software I’ll be building both indi and indi-libcamera from source, but also libXISF which is a dependency for indi that provides XISF support. But let us first install some build dependencies:
sudo apt-get install -y \
git \
cdbs \
dkms \
cmake \
fxload \
libev-dev \
libgps-dev \
libgsl-dev \
libraw-dev \
libusb-dev \
zlib1g-dev \
libftdi-dev \
libjpeg-dev \
libkrb5-dev \
libnova-dev \
libtiff-dev \
libfftw3-dev \
librtlsdr-dev \
libcfitsio-dev \
libgphoto2-dev \
build-essential \
libusb-1.0-0-dev \
libdc1394-dev \
libboost-dev
libboost-regex-dev \
libcurl4-gnutls-dev \
libtheora-dev \
liblimesuite-dev \
libftdi1-dev \
libavcodec-dev \
libavdevice-dev \
libboost-program-options1.74-devNext we’ll be setting up a working folder:
mkdir -p ~/ProjectsLet’s start with building and installing libXISF:
cd ~/Projects
git clone https://gitea.nouspiro.space/nou/libXISF.git
cd libXISF
cmake -B build -S .
cmake --build build --parallel
sudo cmake --install buildNext is indi:
cd ~/Projects
git clone https://github.com/indilib/indi.git
cd indi
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Debug ~/Projects/indi
make -j4
sudo make installGrab a coffee or somethings, this one is going to take a while if you’re like me building it on your RPI. Once done we can check if our indiserver is available with the latest version:
$ indiserver -h
2024-02-01T20:40:10: startup: indiserver -h
Usage: indiserver [options] driver [driver ...]
Purpose: server for local and remote INDI drivers
INDI Library: 2.0.6
Code v2.0.6. Protocol 1.7.Now let’s continue with indi-libcamera:
cd ~/Projects
git clone https://github.com/indilib/indi-3rdparty
cd indi-3rdparty
mkdir -p build/indi-libcamera
cd build/indi-libcamera
cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Debug ~/Projects/indi-3rdparty/indi-libcamera
make -j4
sudo make installWith all of that set and done we’re on to the next step: using our new tools:
Starting the indi server
To be able to connect our host pc to the Raspberry Pi we need to run a indi server on the Pi. We can do so as following:
$ indiserver -v indi_libcamera_ccdIn the output you’ll notice the libcamera driver at work:
2024-02-01T20:54:29: startup: indiserver -v indi_libcamera_ccd
2024-02-01T20:54:29: Driver indi_libcamera_ccd: pid=5997 rfd=6 wfd=6 efd=7
2024-02-01T20:54:29: listening to port 7624 on fd 5
2024-02-01T20:54:29: Local server: listening on local domain at: @/tmp/indiserver
2024-02-01T20:54:30: Driver indi_libcamera_ccd: [3:09:59.402123462] [5997] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
2024-02-01T20:54:30: Driver indi_libcamera_ccd: [3:09:59.593527216] [6003] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
2024-02-01T20:54:30: Driver indi_libcamera_ccd: [3:09:59.604231695] [6003] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx290@1a to Unicam device /dev/media1 and ISP device /dev/media0
2024-02-01T20:54:30: Driver indi_libcamera_ccd: [3:09:59.604426747] [6003] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Telescope Simulator.EQUATORIAL_EOD_COORD
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Telescope Simulator.EQUATORIAL_COORD
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Telescope Simulator.TELESCOPE_INFO
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Telescope Simulator.GEOGRAPHIC_COORD
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Telescope Simulator.TELESCOPE_PIER_SIDE
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Rotator Simulator.ABS_ROTATOR_ANGLE
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Focuser Simulator.ABS_FOCUS_POSITION
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on Focuser Simulator.FOCUS_TEMPERATURE
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on CCD Simulator.FILTER_SLOT
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on CCD Simulator.FILTER_NAME
2024-02-01T20:54:30: Driver indi_libcamera_ccd: snooping on SQM.SKY_QUALITY
Kstars / Ekos client
On your desktop PC you have various indi clients available. I gave Ekos a try. Ekos is a cross-platform client. Open the KStars application:
You can start the Ekos utility by pressing Ctrl + k, or by navigating through the menu via Tools > Ekos. Next a wizard will be started to help you setup your observatory:
Select Next, and on the next step select the remote device option:
In the next window choose Other:
Now enter the IP address of our Raspberry Pi and click Next. PS: I also deselected the Web Manager option here, but more on that later.
And finally enter a profile name and click “Create Profile & Select Devices”:
You’ll be ending up in the Profile Editor window. Make sure to open the dropdown box and select RPI Camera to link the libcamera CCD driver that we loaded to a CCD in Ekos. Press Save.
Ekos is now being started:
At first nothing is shown in Ekos because we haven’t connected to our gear yet. Press the green play button. If you still have your ssh connection open to your Pi from those earlier steps where you started the indi server you’ll now notice a new incoming client connection:
2024-02-01T21:26:21: Client 9: new arrival from 192.168.0.221:42300 - welcome!A new window will pop-up:
In the new window you can toggle the General Info tab to get some insights in the indi driver being at work. In my case it an IMX462 camera, but advertised as IMX290 since that’s how libcamera picks it up.
After pressing the Connect button you get a whole lot of camera settings that you can easily adjust through the GUI:
You may Close this window or minimize it, and once back in Ekos go to the CCD tab. Here you can start your first capture by pressing the camera icon below the sequence box, hovering the icon will tell you “Capture a preview”:
On the Raspberry Pi you’ll now see libcamera being set to work and capture that shot:
2024-02-01T21:51:00: Driver indi_libcamera_ccd: [4:06:30.151699548] [6070] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
2024-02-01T21:51:01: Driver indi_libcamera_ccd: [4:06:30.302132539] [6075] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
2024-02-01T21:51:01: Driver indi_libcamera_ccd: [4:06:30.307911048] [6075] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx290@1a to Unicam device /dev/media1 and ISP device /dev/media0
2024-02-01T21:51:01: Driver indi_libcamera_ccd: [4:06:30.308089276] [6075] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
2024-02-01T21:51:01: Driver indi_libcamera_ccd: Mode selection for 1944:1097:12:P
2024-02-01T21:51:01: Driver indi_libcamera_ccd: SRGGB10_CSI2P,1280x720/0 - Score: 3084.13
2024-02-01T21:51:01: Driver indi_libcamera_ccd: SRGGB10_CSI2P,1920x1080/0 - Score: 1084.13
2024-02-01T21:51:01: Driver indi_libcamera_ccd: SRGGB12_CSI2P,1280x720/0 - Score: 2084.13
2024-02-01T21:51:01: Driver indi_libcamera_ccd: SRGGB12_CSI2P,1920x1080/0 - Score: 84.127
2024-02-01T21:51:01: Driver indi_libcamera_ccd: Stream configuration adjusted
2024-02-01T21:51:01: Driver indi_libcamera_ccd: [4:06:30.313121278] [6070] INFO Camera camera.cpp:1183 configuring streams: (0) 1944x1097-YUV420 (1) 1920x1080-SRGGB12_CSI2P
2024-02-01T21:51:01: Driver indi_libcamera_ccd: [4:06:30.314471895] [6075] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx290@1a - Selected sensor format: 1920x1080-SRGGB12_1X12 - Selected unicam format: 1920x1080-pRCC
2024-02-01T21:51:08: Driver indi_libcamera_ccd: Bayer format is RGGB-12And a preview window will pop-up showing you your first capture!
You can save the preview to FITS, JPEG or PNG on your host pc by pressing the green ‘download’ icon in the upper left corner. Now what’s left for you is to enjoy that first picture that you just have token. At least I hope you have something more interesting than me to capture…
Autostarting indi-server
Until now I’ve been running the indi-server from the shell over an SSH session. Not really the most user friendly approach once you’re in the field, right. But there is indi Web Manager to the rescue. Indi webmanager is python based web application that can start and stop the indi-server for you by means of a REST api call. In lay mens terms it means that you can have the indi-server started by visiting a http web page, sort of. So what’s the difference with starting it over SSH? Well, the indi web manager is supported by Ekos in that it can make the required web calls to setup the indi-server through the INDI web manager. It also allows you to control what drivers need to be loaded. So in other means its also a manager to configure the indi-server plugins. But I found some difficulties installing it, furthermore since my setup is not changing a lot I figured that I didn’t need a daemon controlling our indi-server daemon but could as well create a small systemd service file and be done with it. So let’s go for that option and start creating our own systemd service.
First create a service file:
sudo nano /etc/systemd/system/indiserver.serviceAnd enter following content:
[Unit]
Description=INDI server
After=multi-user.target
[Service]
Type=idle
User=pi
ExecStart=indiserver -v indi_libcamera_ccd
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.targetYou must set the correct permissions:
sudo chmod 644 /etc/systemd/system/indiserver.serviceNow you must reload the daemon lists so that systemd picks up the new service. Only then will you be able to ‘enable’ the service for starting with the OS:
sudo systemctl enable indiserver.serviceThe system will tell you that a symlink has been created.
sudo rebootReboot the system and the indi-server should come up after the reboot. If you’re still experiencing issues you can manually start the service using:
sudo systemctl enable indiserver.serviceNext check the status of the service:
sudo systemctl enable indiserver.serviceIt should tell you that the service is active and running:
Press ‘q’ to quit. You can also inspect the service logs using journalctl:
journalctl -u indiserver.serviceExample output:
Feb 02 22:29:34 rpi2 systemd[1]: Started indiserver.service - INDI server.
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: startup: indiserver -v indi_libcamera_ccd
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: pid=8618 rfd=6 wfd=6 efd=7
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: listening to port 7624 on fd 5
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Local server: listening on local domain at: @/tmp/indiserver
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: [27:45:03.883924469] [8618] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: [27:45:04.001222194] [8623] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider movi>
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: [27:45:04.006897368] [8623] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx290>
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: [27:45:04.007352574] [8623] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/>
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Telescope Simulator.EQUATORIAL_EOD_COORD
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Telescope Simulator.EQUATORIAL_COORD
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Telescope Simulator.TELESCOPE_INFO
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Telescope Simulator.GEOGRAPHIC_COORD
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Telescope Simulator.TELESCOPE_PIER_SIDE
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Rotator Simulator.ABS_ROTATOR_ANGLE
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Focuser Simulator.ABS_FOCUS_POSITION
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on Focuser Simulator.FOCUS_TEMPERATURE
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on CCD Simulator.FILTER_SLOT
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on CCD Simulator.FILTER_NAME
Feb 02 22:29:34 rpi2 indiserver[8617]: 2024-02-02T21:29:34: Driver indi_libcamera_ccd: snooping on SQM.SKY_QUALITYAgain press ‘q’ to quit.
Other options within Ekos
Ekos and Kstart in general offers lot’s of possibilities, much more than I could ever come up with let alone implement them within any reasonable time. You can adjust exposure, set filters, adjust the format, and so on here:
You can also choose were to store the captured file: remote vs local:
And even create sequences with various exposures:
The the button next to the loop icon (which may have it’s icon missing due to a bug) is the one to start a video stream. You can even start recording the video from there:
Stability
I’ve been having some issue with DMA buffers no longer being able to allocate and so on. It always works the first time, but for a second picture or video I end up in trouble and need to reboot the gateway or manually restart the indi-server. So maybe it’s time to bump the libcamera version as well. Currently the Raspbian 12 (bookworm) OS comes with a slightly outdated libcamera v0.0.5 released back in the summer of 2023:
$ sudo apt show libcamera0
Package: libcamera0
Version: 0~git20230720+bde9b04f-1We can update that to version 1.0.0 nowadays if we start from the official Raspberry Pi repo and compile from source. Before we get building we first need to install some more build dependencies:
sudo apt-get install meson
sudo apt install python3-jinja2 python3-yaml python3-plyNow build:
git clonegit@github.com:raspberrypi/libcamera.git
cd libcamera
meson setup build
ninja -C build installThis will take again a considerable amount of time to complete, but if all went well we now have the updated libcamera installed:
pi@rpi2:~/Projects/libcamera $ libcamera-hello --version
rpicam-apps build: f74361ee6a56 23-11-2023 (17:01:08)
libcamera build: v0.1.0+118-563cd78eUnfortunately that didn’t improve anything so I’ll be spending some time to see if we can debug things, but that’s for later.
Processing speed
This was one of the issues I had with my custom SSH script implementation that I wanted to speedup enormously. I was a bit hoping that having everything updated and moving over to a Raspberry Pi 2 would make drastic changes, like maybe 2 or 3 seconds at max for a 1 second exposure shot. I ended up finishing the 1s capture in 8 seconds. But for the 10s shutter I would again end up in over a minute easily. So it’s still far away from what I really wanted! So maybe not needing to open and close the application each time shoves off some time and also moving from a Raspberry Pi 1 to a RPI2 helps a tiny bit here and there, but unfortunately not what I had hoped. So I’m going to have to dive deeper into this matter and get why it’s so slow, do we really have that many parallel things going on here? More and that maybe in a follow up article if I find the time.
The verdict
I’m pleased with the result as all together I didn’t had that much difficulties to set things up. Except maybe for solving one missing build dependency, but that was pretty much it. I’ve had a lot worse build-from-source experiences in the past with other repositories! To my surprise the libcamera implementation is working OK, but the entire libcamera + INDI stack is not yet entirely bug-free. It’s also not yet as fast as I would have hoped for. That’s certainly something I’ll need to further dig into and check with the indi-3rd party team what could be wrong here.
The nice thing about all of this is that now I’m no longer on my own putting things together from scratch. I must say that it’s always fun to just hack yourself something together with a few lines of scripting, but at one point you’re going to have to make the trade-off of doing everything yourselves and spending a lot of time on it versus leveraging on other mens work and making a few mayor leaps forward. With INDI there is now the entire eco-system that’s readily at my hands and I can start exploring maybe adding a motorized mount, or maybe lookout for other desktop clients or maybe even mobile clients so that I don’t have to drag the laptop outside each time. Plenty of options and opportunities now fall within reach thanks to INDI and the open-source community! So I hope to have gotten you inspired to try things for yourself, and leave a line about how you’re experiencing the INDI, libcamera and RPI combination for your astro stuff. Good luck building!
after hours coding 