Adventures Hosting My Site On Gemini

Updated: 2024-07-06

Well, I finally decided: it is time to host this site in Gemini. As noted in How this site is built, it is written in org-mode using org-roam, is part of the Small Web, and thus is a perfect fit….. right?

I should begin with: I find Gemini to be really appealing. I can actually surf Geminispace as I did Gopherspace and the web decades ago. I don’t really surf the web anymore; I search the web for something specific, but the organic discovery of interesting corners is just lacking these days. So I’ve been enjoying Gemini. As one of the people that started preserving Gopher about 20 years ago, it fits right in with my interests.

And thus began quite the adventure.

This site is now available at gemini://gemini.complete.org/

My Gopher site is available at gemini://gemini.quux.org/

Beginning thoughts

One of the recurring themes that you will see here is that many tools are quite buggy and also abandoned. I’m not sure what this says about the community; it seems to still exist and even thrive, but the barriers to entry due to bugginess and abandonware are high.

Generting gemtext

Gemini’s native document format is called gemtext. It is similar to, but not identical to, Markdown. Notably the link format is different, and links never occur inline within a paragraph.

I initially thought, based on a blog story, that I could use ox-gemini to export my org-mode files directly to Gemini. Unfortunately, this didn’t work well at all. For one, the org-mode id: links weren’t resolved (and thus every internal link was broken). Additionally, filenames contained the org-roam date/time component, which is part of the unique identifier but useless and annoying as part of a URL. It couldn’t strip those as I did with ox-hugo and Hugo’s slug. So I pretty soon decided I would go via Hugo for the Gemini site as well.

There are two options for that. The first I tried is gmnhg. However, it had its own Markdown parser, and didn’t respect Hugo’s slug (creating unsightly filenames). Also ox-hugo uses Hugo shortcodes to properly look up the destination for a link, and gmnhg refused to use them, so all the links were really quite broken.

Then I tried Hugo-2-Gopher-and-Gemini. This also required quite a bit of tinkering and had some of the same brokenness and gmnhg. Eventually, since it actually invoked Hugo, I was able to submit a patch to fix the shortcodes. However, this project’s own Markdown-to-gemtext converter was seriously flawed; it would mess up bulleted lists if the Markdown entries for a list contained a newline (valid in Markdown, but not gemtext). It also didn’t work right if I used a dash instead of asterisk for bullets in Markdown. It was basically a regexp converter that was never going to work.

So I figured, OK, there are lots of tools to convert Markdown to gemtext on awesome Gemini; why not use one of them?

Markdown to Gemini Conversion

They also turned out to be largely buggy:

  • gemgen just totally dropped links after the end of long lists.
  • md2gmn maybe got close, but its table rendering was too wide and had few configurable options.
  • lowdown had a number of issues also
  • I wound up using md2gemini, which doesn’t quite format the links like I’d like (I’d prefer the footnote number and link text in the link list). But it was abandoned by its author in 2023 and nobody seems to have taken it up. It works for now, I guess.

So I run things through md2gemini -l paragraph --ascii-table.

Putting together the generation

Back with the hugo-to-Gemini conversion, I don’t use the Python cleanup script; I run Hugo and then use md2gemini and fix it up from there.

There are still some issues; the HTML href targets – written into the document as {#target}, still bled through. And, the /gemini-page.gmi (aka index.gmi) for the generated URLs couldn’t be removed. So…

find public-gg -name 'index.gmi' -print0 | xargs -0 perl -pi \
	-e 's,^=> (/[^ ]*?/)gemini.page.gmi ([0-9]+: /[^ ]*?/)gemini.page.gmi,=> \1 \2,' \
	-e '; s/{#[^}]+\}//'

Hosting it

There are a lot of options for hosting Gemini. For instance:

I required virtual hosting and proxying was a near-requirement, so I wound up with twins.

Now let’s talk TLS. The twins documentation covers self-signed certificates as well as Let’s Encrypt. The Gemini spec recommends clients use Trust On First Use (TOFU). There are a lot of issues with that; if a site rotates its cert, the client will throw an error. So people use very long expiry times (years). Obviously the experience with Let’s Encrypt won’t be great.

So I made my cert. I got twins up and running. It’s reasonably easy. The featureset is good, though while it can log requests using the Apache format, it inexplicably doesn’t log client IP addresses. Weird.

I couldn’t find any server that did full redirects; for instance, to redirect gemini://complete.org/ANYTHING to the corresponding path under gemini://gemini.complete.org/ – nothing could do that. twins could redirect any page under one site to the homepage on another, but couldn’t do a basic pattern redirect.

Hosting the Gopher Site

I actually have two sites: this one, and a Gopher site on quux.org. I saw there is a proxy that lets you browse Gopher from Gemini called agena, written by the originator of Gemini no less. “Great,” I thought. “This will be easy.”

Ha ha. No.

So it turns out that Agena is more buggy than the rest. I had to fix a bunch of things:

  • It assumed it would receive gopher:// requests rather than gemini:// requests. Nope, no client is going to do that.
  • It used self.request.send without checking to see if all bytes were actually sent, resulting in data being randomly lost on different connections. I switched to sendall.
  • It failed on Gopher selectors containing spaces because it neither quoted nor unquoted the Gemini URL versions.
  • Its logging output never had newlines, which made it useless under systemd (and for general grepping too)
  • Gopher mostly predated UTF-8 and a lot of stuff on my Gopher server is ISO-8859-1. Although agena has nominal support for this, it was broken.
  • I needed to lock it down to only proxy my own server rather than all of Gopherspace, and appropriately send gopher:// URLs for things it wasn’t proxying.

Eventually I got it working.

Perhaps I will add Gemini support to pygopherd; it shouldn’t be hard.

EDIT: I didn’t realize Michael Lazar already added Gemini support to pygopherd! Well that simplifies things. Very cool!

Conclusions

This should have been easy. I mean, Gemini is much simpler than the general modern web. But it turned out to be rather more complicated than the small web at least, due a lot of broken and abandoned tooling. A lot of the tooling gives the vibe of not being really tested or finished. I am concerned about the security picture here too.

But, here it is, and I hope to enjoy it!

Gemini is a modernization of Gopher. It is an example of Small Technology. It uses its own protocol and a document format based on Markdown. It is something of a successor to Gopher.

This site is built for modern clients using Small Technology. It is served from static files, which are themselves small. It should make no references to any resources from other servers, which helps protect the Privacy of visitors.