19 September 2021

Weekly Project Update 11

It’s been a week since the alpha release of Feed Journal and so far things are going reasonably well. A handful of people have given it a shot, which is great, and a few of them have provided feedback, which is also great. I’m always a little afraid of getting feedback for things that I release. It may have been one of the reasons why I don’t release anything else I work on, or at least don’t tell people about it. But so far it hasn’t been as bad as I imagined it to be.

Anyway, the work done during the last week was mainly to pay back the technical debt taken out to get the alpha release out the door. There are some changes that I need to make to the SQLite schema around error reporting (the version of SQLite that is being used does not make schema changes easy) and there are some significant bugs that need to be fixed. One of them is what information is stored to track the feed items that have been imported. When an RSS item is imported into Day One, the GUID was meant to be stored in the SQLite database. But I discovered that when the item was being read in from the RSS feed itself, it wasn’t being set properly within the FeedContent class. There is some prioritised fallback logic which chooses the feed item’s URL when there’s no GUID, so I don’t think it’s a major issue, but it’s still a bit of a facepalm moment.

Another thing that will need some thinking about is how much of an import history is maintained within the SQLite database. I was originally going to keep this relatively short, say roughly the last two months if a item hasn’t been seen in the RSS feed. But there has been some feedback about importing older posts or even an entire blog. I’m not sure if it’s possible doing this using RSS (I need to know if there’s a way of paging within RSS) but there are some other ways that this could be done, such as reading a WordPress export format, or maybe just a bunch of Markdown files. This would be nice to have down the line but it makes sense to lay down the groundwork of avoiding duplicate imports into Day One now. Given that blogs can span multiple years, keeping the import history to a couple of months is not going to cut it. Of course letting it grow without bounds won’t work either: it will just slow the application down. I’m thinking of an approach whereby maintaining the last 20,000 or 2 years worth of import records, whichever is higher, might be a decent compromise. I’ll probably need to offer an option in the import screen to say whether to import posts older than 2 years, and if it detects that there are 20,000 import records during a time period that doesn’t quite reach that, it will raise a warning. But I think that’s something for later.

One list thing that I started looking into is how to get the list of the user’s Day One journals to populate the drop-down in the UI. At the moment, the drop-down is being populated from the journal names that the user has typed into Feed Journaler itself. The thinking was that the user entered the journal name once, then if they wanted to use the journal for another feed, they can just select it from the drop-down. But I think using a drop-down here is causing some confusion. I think it implies that the list contains the user’s Day One journals. Actually doing this be ideal, but Day One is not making it easy to get that information. The dayone2 command line does not provide it, and looking around the Day One support site and the source code of other tools has turned up empty. Theoretically this information is available, as IFTTT has a picker for the user’s Day One journal, but I’m not sure as to how I can use this here. I don’t want to start poking into the internal files of Day One itself, as it will tie the app to the internal storage design of Day One, which will make maintenance difficult. I’m not even sure these files are available to me anyway. It looks like many of the other import tools have been abandoned when Day One changed their data storage in version 2. I’ll do some more research into this — maybe look at how the Micro.blog Mac App does it — but it might be that I’ll need to replace the drop-down with a standard text edit with auto complete. This would be so much easier if Day One offered an API.

I also spent some time in Alto Catalogue, just for a change of pace. I finished the little integrated web-player:

Integrated web-playerIntegrated web-player

It works well: no need to manually select the next track to play any more. I added some shortcut keys to reveal the transport controls (p), play/pause (spacebar), goto the previous/next track (, and .) and seek back/forwards by 30 seconds (< and >). It also has a nice scroll effect so that it appears below the nav-bar when the scroll position is at the top of the screen, but will then slowly detach” itself when scrolling down the page.

Anyway, that’s it for this week. Until next week.


Previous post
Weekly Project Update 10 Hey, these updates made it to double digits. I guess dedicating time in your calendar for writing actually work. Big day for Feed Journaler
Next post
Publishing Blot Posts From Github I’m interested in coming up with a technique for publishing Blot posts from computers that are not my own. I previously had a tool which helped me