Update Issues

[adamysu]

I've never been able to successfully update my birdnet-pipy -- it either runs for a really long time with no indication that it's complete, or the containers never come back up. I saw issue #26 that was resolved. Is there still on-going work with optimizing the update script? I am running a Pi 4b

[Suncuss]

Hi, which version you are on curently?

[adamysu]

Currently the latest (v 0.8.4) however getting there was really a lot of trial and error. I've tried to upgrade within the UI itself, and ran into the same thing from issue 26 where it would say updating and then eventualy "Update taking longer than expected. Try refreshing later.". I then SSHed into the box and ran

echo "main" > ~/BirdNET-PiPy/data/flags/update-requested

When i run top, I could see some git stuff running, but after a while it just seems to hang and the containers never come back up.

Next i tried to run sudo ./install.sh --update and basically same thing where it runs and then hangs and never completes. I ended up needing to run the script a few times and it finally completed after a few tries.

[Suncuss]

Which version did you upgrade from? Does this happen every time you try to update to a newer version?

I have a Pi 4 that used to struggle with updates. The update would eventually succeed, but it could take a very long time. The frontend would time out before the update finished, although the system would eventually come back online and work normally.

I introduced a change in version 0.6.0. From that version onward, whenever I push a new commit, GitHub Actions builds and publishes a prebuilt ARM64 container image. When you do update on Pi , it should now pull the prebuilt image instead of building the image from source locally.

That made the update process much faster, especially on the Pi 4 and older Raspberry Pi models, I now rarely run into timeout on older Pi, even Zero 2W seem to be able to handle this.

But i do have backup path, say if the prebuild image cannot be pulled for some reason, in that case, it will default back to build from source.

can you check journalctl -u birdnet-pipy -r see if you can see lines like Jul 14 00:02:44 birdnet-test birdnet-service.sh[283173]: [INSTALL] Pre-built images available, pulling from registry...

[adamysu]

I was upgrading from 0.8.3. and yes, it happens anytime i try to initiate an update. That being said, your description of how updates worked previously seems to describe my situation exactly ... "The update would eventually succeed, but it could take a very long time. The frontend would time out before the update finished, although the system would eventually come back online and work normally." -- it does seem like a combination of retrying and sometimes just letting it run, it does come back online eventually.

It looks like my logs only go back to yesterday which is already after when I did my update so I don't see those lines. I can check the next time there's an update.

[Suncuss]

try see if presse the down arrow key can allow you to scroll down more, usually the logs are kept unless there is a reboot on the Pi OS

[adamysu]

Hey sorry for the delay -- I did reboot the Pi -- I'll try to get logs the next time there's an update.

Separately, from time to time I get a 502 error when I try to access the web interface. In these cases I usually just reboot the Pi, I'm not sure if this is a known issue or something with the way I'm set up. In the future, should I just go ahead and restart the containers on the Pi (assuming I can SSH in) rather than a full reboot? I can also open a separate issue for this if you'd prefer.

[Suncuss]

no worry at all! thank you for helping me to debug this. i have some new commits on the staging branch. so you could give it try, and then monitor it in real time with journalctl -u birdnet-pipy.service -f see if you see [INSTALL] Pre-built images available, pulling from registry...

my latest update on Zero 2W to this version took about 300 seconds. if your Pi 4 is consistently considerably slower than this i would suspect maybe it is more related to SD card? especially you also mention sometime you sees 502 error? I'm curious if you can actually ssh into it when you are seeing 502.

if you can ssh into it, the easiest way would be to just run sudo systemctl restart birdnet-pipy.service

[adamysu]

Yep I see the update available. do you want me to trigger the update by clicking the button in the UI? by running echo "main" > ~/BirdNET-PiPy/data/flags/update-requested ? I'm assuming they do the same thing

[Suncuss]

Great! They are largly the same, but let's do the UI button for this time :D

[adamysu]

It looks like it is indeed pulling the images from registry and completed fine this time in about 15 minutes.

attached full logs

birdnetpi.txt

[Suncuss]

@adamysu thanks for the log! yeah this makes me think it might be the sd card being a lit on the slow side. and if pulling pre-buiid image took 15 minutes, then i can imange it took even much long when build locally in earlier version. I am curious what does the raspberry pi OS sd card test utiliy says.

https://www.raspberrypi.com/news/sd-card-speed-test/

[adamysu]

yeah, i agree its likely the sd speed. looks like i can only use the utility on a newly formatted SD card.

Currently the UI shows "Update taking longer than expected. Try refreshing later." which could be a bit vague -- I'm curious if it'd be possible to show the logs in real time so that the user has some sort of visual indication of progress.