Publishing Blot Posts From Github

I’m interested in coming up with a technique for publishing Blot posts from computers that are not my own. I previously had a tool which helped me here but it’s not working properly and I haven’t looked at it for a while. Also, I was uncomfortable running it on machines that I don’t have full control over, such as my work laptop.

Since I’m using Git for this Blot blog, I’m wondering if simply typing posts directly into Github would work. The workflow I’d like to try is:

  • Creating new files in Github and using the builtin web editor.
  • Committing the changes to the main branch once they’re ready.
  • Using Github Actions to automatically push the changes to Blot.im

Setting up the Github repository was relatively easy. Once I created it, I renamed the existing Blot origin to blot, and changed origin to the new Github repository:

git remote rename origin blot
git remote add origin git@github.com:lmika/random-development-blog.git
git branch -M main
git push -u origin main

Adding a workflow to publish to Blot when pushing to main took a bit longer, but after some trial and error (and many emails from Github about failed action runs), I eventually got a workflow which does this:

# Filename: .github/workflows/publish.yml
#

name: publish

on:
  push:
    branches:
      - main

jobs:
  publish-to-blot:
    name: Publish to Blot
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Setting up credentials
        env:
          BLOT_LOGIN: ${ { secrets.BlotLogin }}
        run: |
          git config credential.helper store
          echo "https://${BLOT_LOGIN}@blot.im/" >> $HOME/.git-credentials
      - name: Push to Blot.im
        run: |
          git remote add blot 'https://blot.im/clients/git/end/therandomdevelopmentblog.git'
          git push --force blot main:master

Note: There’s a bug in Blot which does not allow for two open curly braces, so that space between the { for the BLOT_LOGIN environment variable does not actually appear in the file.

Having the fetch-depth: 0 in the checkout action is important. By default, this action creates a shallow clone of the repo, but Blot needs to full clone or it won’t accept the push. I also had to include my Blot Git credentials some way. This was easy to do using Github repository secrets:

  1. Go to the Settings” tab in Github.

  2. Go to the Secrets” tab.

  3. Create a new secret with the name BlotLogin”

  4. Set the value of the secret to <username>:<password> where username and password are the ones generated by Blot. If the username is an email, which it is for me, then the @ sign will need to be encoded as %40 so that it looks like:

    user%40example.com:password

We’ll see if this works as a way of getting posts into Blot. I guess at the very least, it’s good to have an extra copy of this blog.

19 September 2021 Techniques

Weekly Project Update 11

It’s been a week since the alpha release of Feed Journal and so far things are going reasonably well. A handful of people have given it a shot, which is great, and a few of them have provided feedback, which is also great. I’m always a little afraid of getting feedback for things that I release. It may have been one of the reasons why I don’t release anything else I work on, or at least don’t tell people about it. But so far it hasn’t been as bad as I imagined it to be.

Anyway, the work done during the last week was mainly to pay back the technical debt taken out to get the alpha release out the door. There are some changes that I need to make to the SQLite schema around error reporting (the version of SQLite that is being used does not make schema changes easy) and there are some significant bugs that need to be fixed. One of them is what information is stored to track the feed items that have been imported. When an RSS item is imported into Day One, the GUID was meant to be stored in the SQLite database. But I discovered that when the item was being read in from the RSS feed itself, it wasn’t being set properly within the FeedContent class. There is some prioritised fallback logic which chooses the feed item’s URL when there’s no GUID, so I don’t think it’s a major issue, but it’s still a bit of a facepalm moment.

Another thing that will need some thinking about is how much of an import history is maintained within the SQLite database. I was originally going to keep this relatively short, say roughly the last two months if a item hasn’t been seen in the RSS feed. But there has been some feedback about importing older posts or even an entire blog. I’m not sure if it’s possible doing this using RSS (I need to know if there’s a way of paging within RSS) but there are some other ways that this could be done, such as reading a WordPress export format, or maybe just a bunch of Markdown files. This would be nice to have down the line but it makes sense to lay down the groundwork of avoiding duplicate imports into Day One now. Given that blogs can span multiple years, keeping the import history to a couple of months is not going to cut it. Of course letting it grow without bounds won’t work either: it will just slow the application down. I’m thinking of an approach whereby maintaining the last 20,000 or 2 years worth of import records, whichever is higher, might be a decent compromise. I’ll probably need to offer an option in the import screen to say whether to import posts older than 2 years, and if it detects that there are 20,000 import records during a time period that doesn’t quite reach that, it will raise a warning. But I think that’s something for later.

One list thing that I started looking into is how to get the list of the user’s Day One journals to populate the drop-down in the UI. At the moment, the drop-down is being populated from the journal names that the user has typed into Feed Journaler itself. The thinking was that the user entered the journal name once, then if they wanted to use the journal for another feed, they can just select it from the drop-down. But I think using a drop-down here is causing some confusion. I think it implies that the list contains the user’s Day One journals. Actually doing this be ideal, but Day One is not making it easy to get that information. The dayone2 command line does not provide it, and looking around the Day One support site and the source code of other tools has turned up empty. Theoretically this information is available, as IFTTT has a picker for the user’s Day One journal, but I’m not sure as to how I can use this here. I don’t want to start poking into the internal files of Day One itself, as it will tie the app to the internal storage design of Day One, which will make maintenance difficult. I’m not even sure these files are available to me anyway. It looks like many of the other import tools have been abandoned when Day One changed their data storage in version 2. I’ll do some more research into this — maybe look at how the Micro.blog Mac App does it — but it might be that I’ll need to replace the drop-down with a standard text edit with auto complete. This would be so much easier if Day One offered an API.

I also spent some time in Alto Catalogue, just for a change of pace. I finished the little integrated web-player:

Integrated web-playerIntegrated web-player

It works well: no need to manually select the next track to play any more. I added some shortcut keys to reveal the transport controls (p), play/pause (spacebar), goto the previous/next track (, and .) and seek back/forwards by 30 seconds (< and >). It also has a nice scroll effect so that it appears below the nav-bar when the scroll position is at the top of the screen, but will then slowly detach” itself when scrolling down the page.

Anyway, that’s it for this week. Until next week.

19 September 2021 Projects

Weekly Project Update 10

Hey, these updates made it to double digits. I guess dedicating time in your calendar for writing actually work.

Big day for Feed Journaler yesterday. Have got an early access release out the door. I’ll admit, it took some getting over myself to getting it out there. This whole announcement” thing is not something I’m good at: I generally don’t like getting a lot of attention (not that there was a lot of attention). But besides that, it was notable to some extend: it’s my first distribution of a Mac app and I think my third real distribution of a finished” product.

There was a flurry of activities that lead up to this: getting the website up, fixing the last few bugs — including one show-stopper which was not properly changing the feed configuration view controller after restoring the window — and getting the app signed by Apple.

Here’s how the app looks now:

Main window of feed journalerMain window of feed journaler

There’s still a lot to do — this is only an early access release (i.e. alpha release) so there are several things that are still outstanding. But for the moment, I’m taking a little break to work on Alto Catalogue for a day or two.

I’ve been finding myself using it more often for my music listening, even though the same songs are on my local disk. It’s just that little bit nicer to navigate through my collection there than digging through the file system (I wouldn’t call it a nice experience though, it’s still pretty basic). But the music listening experience is pretty crappy. Clicking the play button next to the track brings up the browser-provided audio player in a separate tab, and although that works for single songs, it would be nice if the player plays through an album. I’m in the process of adding a basic web-player to achieve this. Once it’s done, the only issue would be that going to a new page will stop playback since the HTML frontend is being statically rendered and the the whole page is reloaded when clicking a link. There’s no real easy way to get around this, apart from rebuilding the entire UI in something like React, something that I’m not too keen on doing at this stage. There is hotwire.js which might help. Maybe there’s a thing there that would allow the page to reload except for a particular part of the DOM. I’ll probably need to spend some time looking into this.

Anyway, not heaps to say this week. Until next week.

12 September 2021 Projects

Weekly Project Update 9

Day One Sync has a new name. Turns out that there is already something offered by Day One the company (I guess it’s now Automattic) called Day One Sync, so using this name is no longer possible. It was never really a good name anyway.

So now the app is called Feed Journaler.1

The only downside of this rename though is the app icon. The prominence of the lowercase D in the design was based on the idea of Day One” being somewhere in the name. It may have to do for the moment: I’m not too keen with changing it at this late stage. Maybe it won’t matter in the end: it could take on a more abstract representation of what the app is doing. That’s probably not good iconography design, but I won’t say anything if you won’t. 🙂

Anyway, this last week was spent on closing out the many of the remaining niggly things. There was actually a few more than I realised, like making sure the proper feed is displayed when creating or deleting feeds, making sure that the window is restored properly when closing it and clicking the dock icon, and just making the UI more usable. The feed status display needed a bit more work than I thought as well. Ideally, when the app encounters an error fetching and importing a feed, the status display should reflect that. This was previously not happening, and the status Up to Date” was being shown even though some feed items failed to be imported. I’ve started working trying to address this. I’ve got the foundations there, but it may be that some additional work will be required down the line to actually show which feed item failed, why they did, and how they can be resolved. It’s likely that some additional UI will need to be built.

There are still one or two small things that need finishing, but apart from that, I think this release is more-or-less feature ready. It’s not pretty — I’m not going to win any design awards for this — but I hope that some of the rough edges have been smoothed out. There are still plenty more, but that may have to wait.

Now comes the fun part: distribution.

I’m hoping to make this release available in some form of public early access program. The app is currently not sandboxed — it needs to run a command that comes from Day One, which I’ve been unable to do from within an app container — so the Mac App Store is out of the question. So there will be time dealing with certificates and profiles and all the other stuff Apple requires developers to do to get their app out there. Doing this manually is okay so far. I’d like to automate this in some way down the line, but I get the feeling that stepping away from the paths of Apple even slightly will mean a lot more work. Doing it all in Xcode by clicking things will have to suffice for the moment.

Instead, I’ll need to stand up a website to through which this app can be made available to others. Ironically, some of the very last things left to do in the app itself requires the website to be built: there are a few buttons and links which need to open up the website in a browser. It may take time standing up the Sparkle framework to handle automatic updates, so it’s likely that this 0.1 release will not have this. So I’ll also need some way of announcing new releases when they’re available. For a little while I considered setting up a mailing list, but I’m not sure I’m comfortable with that at the moment. My current thinking of doing this in the form of RSS blog-posts, that those currently using the early access release can subscribe to in their feed reader. Maybe also a Twitter handle as well.

So that’s my focus for the coming week.


  1. It might be that this name is no better: coming up with good names is hard. At least the domain name was available.↩︎

5 September 2021 Projects

Weekly Project Update 8

It was difficult writing this update this week. I have to stop reading the news before I do.

Anyway, I spent a bit of last week working on Day One Sync. It feels like I’m in the last phase of this first release. I spent most of the time working on improving the usability of the UI, with a fair bit of that effort trying to get the constraint based layouts looking good. The feed update status is now working, and is being dynamically updated when the feed is being fetched in the background, which fixes the problem of having to switch between feed configs to get updated status information. In addition to this, another problem involving the resizing of the text view showing the summary of the last fetched item has been fixed. Previously this was resizing the whole window, but after reducing the content compression resistance priority from 750 to 250, the window remains at the same size and the summary is being neatly truncated.

Also of note is that feed names are now being set from the feed itself. After changing the feed URL, the app will now fetch the feed and set the name from the title. I originally had the idea of making the name configurable, but I figured that might be too much for an initial release.

Finally, the delete toolbar button is now working, making its possible to delete feed configs once more. This took some time to work out, and the solution that I’m using is not really all that nice. My main issue is that instantiating a new view controller when changing the selected feed config does not add that view controller to responders list. This is affecting the delete toolbar and menu item, which was sending a deleteSelectedFeed message to the first responder, but was not getting picked up by anything. I’ve got a workaround in place using the message bus library. It works but it’s not perfect as the delete feed menu item is still available even though no feed is selected. I think I’ve got a fix for this as well that won’t be pretty, but may have to do for the time being.

I think a lot of the work remaining are the small, niggly things that are relatively straightforward to do. It’s just a matter of finishing them. The thing is that as I approach the finish line of this project, I start getting reluctant to cross it. I’ll admit it, there’s an element of fear involved with releasing this. It’s not so much a fear of failure: that I can handle, as it wouldn’t be the first time I’ve released something that goes nowhere. It’s a little more of a fear of success, and having something that will be used and that would need supporting. I know that sounds a little presumptuous, but what can I say? It’s a real feeling I have. It might be that I’m thinking too much of that.

Anyway, that’s the update for this week. Until next week.

29 August 2021 Projects

Weekly Project Update 7

It’s probably just as well that this weekly post is mainly a public journal entry, rather than much of anything else. It feels like I didn’t do a lot of goal driven work this week. Everything done was mostly for the sake of doing something, mainly as a distraction to deal with the news. That said, there are some things that could be reported this week.

For Day One Sync, this week included a mix of frontend and backend work. The new Markdown to HTML translation code has been integrated and some work to clean up the code has also been done. Also, work to include the status of the last feed fetch in the UI has begun. Below the feed configuration is a small field displaying whether the last poll from that feed was successful, as well as what the last RSS feed item read actually was. Previously these labels were just placeholders but since that this information is now being recorded when the feed is polled, it can be displayed to the user when they open up that feed configuration in the UI. There are still some issues with the display. The first is that the status is not updated when the feed is being polled while the configuration is in view. To get an update, the user will need to switch to a different feed configuration, and switch back to get the updated view. I think I’ve got a solution to this involving the event bus and I’ve sort of started work on that, but it’s still early days so far. The second issue involves auto-layout. When the title of the last read feed item is long, the calculated width of the label pushes out the size of the window, making it wider. I’m generally not a fan of this sort of behaviour. I would prefer the window size remaining unchanged, and the label value being clipped. I’m still looking into how I can fix this.

I also spent some time on another project this week, just to provide some variety. I alluded to this project last week but I think it’s fair to write a bit more about it since I’ll be talking about it. This project, called Alto Catalogue, is a self-hosted music locker service for private music stored in S3. It consists of a web-app written in Go using the Buffalo framework that has been release as open-source, and an Android mobile client which hasn’t been released at all. I’m generally hesitant to write about this project as it’s in a bit of a frustrating state. It was one of those projects that was built for my own use, meaning that there are a lot of shortcomings that I myself can live with, but are a bit embarrassing if I were to share it with others. It might be that these limitations means that releasing this as open source makes little sense, which is a fair enough argument. But I guess another way to look at it is that the overall effect of open sourcing this now, with no-one looking at it, has the same overall impact as keeping it close. I guess you could call it the tree-falling-in-the-woods” decision.

Anyway, I did some work on this project this week around adding group headings to tracks. This is especially useful for albums that would traditionally be released on multiple CDs, such as the most recent release of Oxygène, which features three disks, each one a distinct volume. Having a long track listing, all of them with names of the form Oxygène X” where X is some number, makes it difficult to tell where these volumes actually are.

This was the main motivation for adding group headings, which can be defined on the first track of each album to provide some ease of locating the divisions. How this was implemented in the catalog service is also slightly frustrating, but could end up being useful. I originally considered using metadata for this, but the metadata system has some pretty large limitations related to its design. The first is that it is unstructured: you can define a metadata field with a key or value of your choosing, and no structure over the data, like requiring it to be in JSON, is imposed. This leaves the use of the metadata system largely undefined, which is great for allowing a variety of use-cases, but it does mean more work in the Android app to parse this metadata. That in and of itself wouldn’t be too much of an issue if it wasn’t for the second limitation, which is that fetching metadata is an extra HTTP request. This means that when the track list is refreshed, not only would the track details need to be fetched, but also all the track metadata as well. Given that this grouping would be used in the track list UI of the app, this will slow the process down.

So I wanted this information stored against the track. But I wasn’t too keen of having this stored in a dedicated database column as I’d imagine it would only be used by a few tracks. So I settled on a middle way, which was adding a JSON attributes” field on the track. The field is meant to store additional” information about the track, but unlike metadata, is not meant to be too free-form in what it is used for. I’d like to keep the field naming relatively controlled so that it could be used appropriately by clients. This is something that I explicitly decided against for metadata. The attributes field is exposed as a JSON text editor in the UI, but I can envision it being backed by a proper form base UI as well, another thing that would be difficult to do with metadata. Track group names is the only thing this new field is being used for at this stage, but I can see it being used for storing other things about the track, like track descriptions, recording date, artist if they differ from the album artists, country and year if the track is a Eurovision song, etc. We’ll see how it goes. If it proves useful, I may also add it to the album type.

22 August 2021 Projects

Weekly Project Update 6

Oh dear. I wish I had a decent update to write this week. The truth is that not a lot of what I’ll call deep work” was done. I’ve been spending some time on other hobbies and projects, probably to add a bit of variety to my evenings. Work is still progressing on Day One Sync” but mainly in the core of the app, with much of it improving the quality of the code and doing a bit of refactoring. I think I’ve got most of the HTML -> Markdown translation working though, and at the moment it’s just testing it. Some very minor work was done to the UI as well, such as adding the ability to tab between fields, but nothing of substance that is worth reporting.

I did have the opportunity to work on another project that has been brewing on the side for the pass year or so. I haven’t got much to say about it except that there are some fundamental usability tasks that need doing that I have been putting off for a while. Such as with many of the projects I start, I tend to get it to a state that I deem usable for myself, and then loose interest in going further. Usually a project in that state is not one that I deem suitable for sharing with others. Many times I see little point in filling these gaps, which is absolutely the wrong attitude to have if I’d like to get it to state that others would want to use it. I’m guessing that the reason for this is a matter of perspective: as long as the project is in a state that I can use it, then that’s good enough.

In an attempt to counteract this a tad, I’m trying something a little different. I’ve started writing some user documentation at the same time, and it has occurred to me that doing this acts as a decent method of finding out where the holes are. When I try to explain how to setup a server (this project is a web-app) or do a common thing within it, and I find that I can’t do so without writing caveats or explaining why this can’t be done at all, then that’s a good indication that improvements need to be made. Of course, the underlying drive to write this documentation is no different than the drive to improve the user experience for others, bringing us back to the perspective problem once again. I wonder how others handle this.

Anyway, that’s all for this week. Hopefully I’ll have something more to write about by next week.

15 August 2021 Projects

Weekly Project Update 5

I think writing these updates are helping me stay disciplined to work on this project. There have been times during the last few weeks when I felt like working on something else, and there has been a day or two that I’ve done just that, or have not worked on anything at all. But I think the unwillingness to arrive at this scheduled post time on Sunday with nothing to write about is motivating enough to actually get some work done.

On to this week’s update.

Day One Sync

Slightly shorter post this week as the details of what I worked on this week would be slightly tedious to write about (and probably read about). I spent the week working on the UI a bit, mainly on finalising the layout of the feed configuration details view. I’ve converted the UI from springs-and-structs to a constraint-based layout. Getting this working took a bit of effort, and I’m a little concerned about how easy it would be to maintain and modify this UI going forward, but for the moment I’m reasonably happy with this layout.

Day One Sync window with the new layoutDay One Sync window with the new layout

The engineering behind the UI is still outstanding. At present, only the feed URL, journal name and tags controls work. The checkbox to download images is being save for the feed, but the actual value is currently being ignored. The rest of the controls do nothing, and getting them working will probably be the last things I do for the UI, just because of the fiddly details involved in updating them dynamically. I think the libraries I’m using for HTTP and events would help here. As you can see, I haven’t made any changes to the sidebar yet.

I said I wouldn’t work on the core last week, but after dealing with the UI layout, I needed to work on something that felt a bit more like coding. The HTML-to-Markdown converter needs a fair bit of work. At present, the tool is parsing the HTML content from the RSS feed, and converting it directly into a Markdown string on the spot. This sort of works, but it’s a bit of a hack job and maintaining it going forward would be difficult. It’s also inflexible: if I wanted to change the generated Markdown in some way, I’d have to hack it up in the generator, making it even harder to maintain.

So I spent some time last week working on an intermediate representation of the generated Markdown that can be built from the HTML parser. This intermediate representation of the generated Markdown is just an object graph, making it relatively easy to modify one it is built, and it itself can be used to generate the Markdown string. This essentially changes the processing flow from HTML → Markdown, to HTMLIR → Markdown. Already this is producing a more correct version of Markdown with support for things like ordered lists, something that the existing Markdown generator does not handle. This is not quite finished, and I need to integrate it with the HTML parser, but I’m reasonably confident that this wouldn’t be too involved.

Along with the work done above, I’m doing a bit of dog-fooding by importing posts from the various feeds I publish into Day One. Now with the IFTTT integration disabled, this is the only way that I can get my blog posts into Day One, so I have an incentive to make sure this tool works. The posts are just going into a test journal at this stage, and I’m manually moving the posts I’d like to keep to the production” journals once they’re imported. I’m not really stressing it at this stage, but so far no major issues apart from the lack of full Markdown support for things like lists.

That’s all for this week. Until next week.

8 August 2021 Projects

🔗 How Shopify Uses WebAssembly Outside of the Browser

A fascinating post on how Shopify is using WebAssembly as a runtime container on the server. References some interesting technologies like Lucet and AssemblyScript.

1 August 2021 Links

Weekly Project Update 4

No preamble today. Let’s get straight to the update.

Day One Sync

The core of the app is coming along. Feeds are being polled and new items are being added to Day One using the command line tool. I’m in the process of testing this on my own blog posts. I’ve disabled the IFTTT RSS-to-Day One integration and I’m now relying on this to harvest these posts and add them to Day One. It’s already a massive improvement over the old system. For one thing, the formatting of blog posts is preserved, meaning I no longer need to fix this myself. The tool is also uploading any images included in the blog post to Day One: no more weird links to IFTTT in place of images. The core is by no means complete, and it’s also an ugly mess that needs a fair bit of rework, but it’s probably something that could be set aside for a little while.

Now comes the hard part: making the UI. I say making the UI because what is in place at the moment is only barely functional. It’s just enough to get the feed polling stuff working, but it’s not in a state that I’d like to share with others, or even use myself. However, I’ve made the first steps towards that, with replacing the main view controller with a SplitViewController. It was less difficult than I expected, and it’s nice separating concerns across multiple classes. It does make communication across the view controllers a little more difficult. I’d rather not have view controllers know about each other, as that just increases the coupling between them, making the code difficult to work with down the line. Instead, I’m opting for an event-based approach to do this, and I’ve found a relatively simple library, called SwiftEventBus, to do this.

I’m hoping for a UI that looks at home on Big Sur, complete with the split toolbar seen on apps like Mail and XCode. Thanks to this WWDC video and this blog post, I managed to achieve this yesterday. I must say that it provides a nice way to separate the toolbar items based on what they operate on: the buttons on the left operating on the list of feeds, and the buttons on the right operating on the feed itself. I had a bit of trouble getting the NSTrackingSeparatorToolbarItem aligned with the segmented control though. Many of the examples set the dividerIndex to 1, which I guess works on a three-column window. This app only has two columns though, and it was only when I set dividerIndex to 0 did the divider nicely align with the split view. Also, given that this class is not available in the storyboard palette, I had to change the toolbar so that the items are now being created in code. This may end up being an advantage though, as I’d like the items on the right to change based on the controller being shown in the details view.

One last thing that took me a while to figure out is how to send events to the first responder from code. Since the toolbar items are being created in code, I couldn’t rely on the connections defined in the storyboard. I’d also didn’t want to add the IBAction to the window controller as I could imagine that being unwieldy as more actions are added over time. So I’m using the pattern described in this Stack Overflow answer, which involves using a selector for a function definition on a protocol. My first attempt at this was producing the error unrecognized selector sent to instance. It was only after reading this blog post that I realised I was setting the target incorrectly. It was being set to the window controller, which did not implement the protocol. Setting the target to nil resolved the issue, and I was able to send events that were being picked up by view controller in the responder chain that implemented this protocol.

This is now working for adding new feeds. I now need to make it work for removing feeds, which I’d like to be handled by the view controller representing the feed. This view controller is instantiated in code when the selected feed in the outliner is changed, but it does not seem to be included in the responder chain, and the events to delete the feed are being lost. That will be the next thing I tackle.

That’s all for this week. Until next week.

1 August 2021 Projects

Weekly Project Update 3: Ripping The Guts Out

I was a bit more productive this week than I was last week. It seems like the balance of two forces acting on my productivity — the anxiety of the current lockdown, and the boredom that comes from running out of things to do at work (I will be changing jobs soon) — have shifted slightly towards the get-stuff-done side. This can only be a good thing, and I hope it will continue into the coming week.

Of course, a rather impact on the progress of an app is the amount of rework that comes from figuring out a better way to do something. That’s the trouble with building something with technologies that you’re learning at the same time. Getting something to a state that it just sorta’ kinda’ work, then realising that due to some limitation (cough sandbox cough), your entire approach is faulty and large chunks of the application will need to be redesigned to accomodate it. Sometimes I wish for nothing less then ripping the guts out completely, and starting over from scratch.

Anyway, here is this week’s update.

Day One Sync

Ok, Core Data is out. I’m sure it’s a great framework but there are too many new things that I need to learn at the same time, and using it will just slow me down. So instead I’ve opted to using SQLite.swift, which provides a nice Swift interface to a Sqlite database. Porting the persistence logic to this new library has made it so much simpler; gone are the DTOs, and the need to keep multiple classes in sync. I’m still unsure as to whether I should use separate Sqlite files for the feed configuration, and to track which feed items were added to Day One. I think for the moment I’ll keep it in a single database file. It will be much simpler and I can rely on the niceties provided by Sqlite like cascading deletes.

The mechanism of fetching items from an RSS feed and adding it to a Day One journal is coming along as well. I have got it to the point where new items from the two test feeds I’m using are being added to the Day One, complete with formatting and attached images. It’s still early days now: the logic needs a great deal of cleanup. Also, my assumptions that I could just import the HTML directly from the feed items proved wrong: doing that resulted in weird journal entries which seemed to destabilise Day One. It looks like Day One prefers entries added as Markdown, so the current work here is to write a HTML to Markdown translator. I’ve made a start to this, but I imagine that this will be a bit of a fiddly job that will require a fair bit of testing.

As for adding the entries using the dayone2 app, I hit a bit of a brick wall in how I could do this with a sandboxed app. I’m hesitant to bundle the dayone2 app within my app. It’s not my code and I have no idea if the app would be different for each install of Day One. I tried allowing the user to select the dayone2 app from a file picker, but that didn’t seem to work. It may work if I granted full file system access to Day One Sync, but I’m not sure how to easily test this from Xcode builds.

So at the moment, Day One Sync is running without a sandbox. This is not ideal, and maybe near the end, when I start creating standalone archives I can test a sandboxed app with full disk access. It seems to me that requesting permission to run apps within /usr/local/bin, which as far as I know are binaries that should be available to anyone, should be supported as part of the sandbox. As it stands at the moment, it seems quite limited as to what a sandboxed app can do (I guess that’s by design but I know I’m not the only one that thinks this).

Also needing a lot of work is the UI. I realised that I made a mistake when I made the main window out of a ViewController with a splitter view. I realised now that I should have used a SplitViewController. This will give me the master-detail splits in the title bar that seem to be a hallmark of Big Sur apps. So I guess more work will need to go into redoing all that. It’s not much of a problem though: the UI needs a lot of work anyway so it’s good catching this sooner rather than later.

That’s all for this week. Until next week.

25 July 2021 Projects

Weekly Project Update 2: Nothing’s Finished

I didn’t spend a lot of time working on side projects this week. News of the Covid-19 outbreak in Melbourne was weighing on my mind. I guess at this stage there’s no real need to work on them: this is currently still a hobby, and it’s not like there’s a deadline associated with them.

But I have to acknowledge that there’s a downside to thinking this way, particularly if I want to make this more than just a way to kill time. After all, in order to share something with others, it has to be in a state that could be loosely defined as finished”; and at the moment, none of the projects I’m working on is really in this state. This is a bit of a problem of mine: it’s always more fun to start new projects than it is to follow through and polish them up, particularly when building something for yourself.

I’m not quite sure how I would be able to work through it though. Maybe seeing someone post about a problem that I’m trying to solve myself would motivate me to actually get something to a shippable state. Maybe applying rules like no new projects until I finish the one’s I’m working on new” is the way to go, although past experience with trying this has not been super successful. Maybe writing these reports would help.

Anyway, on to this weeks update.

Day One Sync

Despite doing a little this week, I did manage to get some work done on this project, mostly on the backend” (i.e. not the user interface). In the last update, I mentioned that I upgraded my OS to Big Sur in an attempt to get a newer version of Xcode with the hope of adding SwiftSoup as a Swift package. Well, I tried that, and I was still seeing the same error. I didn’t want to waste any more time on it so I decided to move the dependencies to CocoPods, which worked without a hitch. It’s a shame that I can’t use the official” facilities that Apple is building, but I guess CocoPods has been around for much longer, and will probably have better community support anyway.

The other major work done this week was getting Core Data working. This will be used to store the configuration options for the feeds that will be polled. The entities were already been defined but before this week I was unable to get anything written to the store. I was having trouble in how I could deal with the managed objects in the view controllers outside a persistence context.

I think I’ve found a solution to this, and although it works, I already feel that it’s the wrong one. It’s one inspired from a pattern I used when working with, of all this, enterprise Java apps. Basically, I have two classes for each entity: one which is managed by the persistent context, and another which is just a plain Swift class acting as a DTO. While there are strict rules regarding how to work with entities, the DTOs themselves can be worked with like any other type: there’s no magic to them at all. The fields of a DTO can be copied to an entity, and visa-versa, making it easy to separate concern: I can keep the entities in a repository class that has methods that take and return DTOs.

I’m probably making this more complicated than it needs to be, most likely because my knowledge of Core Data works is relatively limited. It’s also probably my experience with working with other ORMs, and the associated magic” that comes from that. Magic in your application is a double-edge sword: it might make it easy to develop, but you generally pay the price when things go wrong, and you have no idea how to resolve an issue. At least with DTOs, the Core Data aspect of the application is localised in a single location.

One other thing I’m considering Core Data for is to keep track of the posts that were imported. This is to prevent double entries, since the Day One CLI doesn’t have a way to give that information to you. I’m wondering whether it would be worth keeping that information in a separate store. I haven’t explored how multiple stores work with Core Data, and I hope that there are no hidden gotchas associated with doing it this way.

That’s all for this week. I hope to get something written to Day One by the time of the next update.

18 July 2021 Projects