From 942af17f577e5017340cf7091353ce5dab1a9f47 Mon Sep 17 00:00:00 2001
From: John-Mark Gurney <jmg@funkthat.com>
Date: Wed, 7 Dec 2022 12:24:52 -0800
Subject: [PATCH] update blog post now that snapaid.sh makes things easier...
add ability to update blog posts, missing is proper redirect from the
old version.
---
.../2018/07/making-freebsd-magnet-links.html | 3 +
.../2022/12/making-freebsd-magnet-links.html | 172 ++++++++++++++++++
layout/blog.j2 | 3 +-
3 files changed, 177 insertions(+), 1 deletion(-)
create mode 100644 content/2022/12/making-freebsd-magnet-links.html
diff --git a/content/2018/07/making-freebsd-magnet-links.html b/content/2018/07/making-freebsd-magnet-links.html
index 115426d..9089f81 100644
--- a/content/2018/07/making-freebsd-magnet-links.html
+++ b/content/2018/07/making-freebsd-magnet-links.html
@@ -9,6 +9,9 @@ tags:
- magnet
---
+<blockquote><span style="font-size:2em">⚠</span>️ This post is old, the most up to date one is
+<a href="/2022/12/making-freebsd-magnet-links.html">here</a>.</blockquote>
+
For the last few years, I've been producing torrents and publishing
magnet links, but there is some special work that I do to make these.
The first few releases, I inserted a bogus tracker into the torrent,
diff --git a/content/2022/12/making-freebsd-magnet-links.html b/content/2022/12/making-freebsd-magnet-links.html
new file mode 100644
index 0000000..3904ffa
--- /dev/null
+++ b/content/2022/12/making-freebsd-magnet-links.html
@@ -0,0 +1,172 @@
+---
+title: Making FreeBSD magnet links
+description: >
+ Making FreeBSD magnet links
+created: !!timestamp '2018-07-03'
+time: 4:49 PM
+updated: !!timestamp '2022-12-07'
+updated_time: 12:20 PM
+tags:
+ - FreeBSD
+ - magnet
+---
+
+Note: This was originally posted
+<a href="/2018/07/making-freebsd-magnet-links.html#making-freebsd-magnet-links-title">here</a>,
+but things have been improved since then and so it has been updated.
+
+For the last few years, I've been producing torrents and publishing
+magnet links, but there is some special work that I do to make these.
+The first few releases, I inserted a bogus tracker into the torrent,
+because despite there being plenty of tools out there for producing
+trackerless (DHT) torrents, they were all GUI and I never found any that
+were command line based. The other was there was/is no tool for
+extracting the info hash and building the magnet link. There may be
+tools now, but I couldn't find any when I started 3 years ago.
+
+The following steps are based upon the recent release of FreeBSD 11.2‑R,
+adjust as necessary.
+
+<ol>
+<li>Fetch the FreeBSD release and release announcement into a directory
+ (I create a per release directory).
+ remove the <code>CHECKSUM.SHA256</code>, <code>CHECKSUM.SHA512</code> and <code>index.html*</code> files.
+ <pre class="fullwidth"><code>
+ $ REL=12.4-RELEASE
+ $ RELABV=12.4R
+ $ curl https://www.funkthat.com/~jmg/FreeBSD-snap/snapshot.idx.xz | xzcat | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3 wget --no-verbose -c --limit-rate=7000k
+ $ wget https://www.freebsd.org/releases/$RELABV/announce.asc
+
+ </code></pre>
+ This step depends upon
+ <a href="https://funkthat.com/gitea/jmg/snapaid">snapaid</a> running.
+ If it is no longer running for some reason, <code>addinfo.sh</code> could
+ be run manually against the release email (not the downloaded
+ <code>announce.asc</code> file as the message id is needed) to generate
+ snapshot.idx.xz locally, and use that instead.
+<li>Verify the GPG key that signed the above files. This is usually Glen
+ Barber's key, but not always. I have met and verified his fingerprint
+ in person, If you have verified someone's key who has signed Glen's
+ key, that is another good way.
+<li>Verify the release announcement:
+ <pre class="fullwidth"><code>
+ $ gpg --verify announce.asc
+ Warning: using insecure memory!
+ gpg: Signature made Mon Dec 5 17:34:59 2022 PST using RSA key ID 478FE293
+ gpg: Good signature from "Glen Barber <gjb@FreeBSD.org>" [unknown]
+ gpg: aka "Glen Barber <gjb@keybase.io>" [unknown]
+ gpg: aka "Glen Barber <gjb@glenbarber.us>" [unknown]
+ gpg: aka "Glen Barber <glen.j.barber@gmail.com>" [unknown]
+ gpg: WARNING: This key is not certified with a trusted signature!
+ gpg: There is no indication that the signature belongs to the owner.
+ Primary key fingerprint: 78B3 42BA 26C7 B2AC 681E A7BE 524F 0C37 A0B9 46A3
+ Subkey fingerprint: 8D12 403C 2E6C AB08 6CF6 4DA3 0314 58A5 478F E293
+ </code></pre>
+<li>In the past I have used BitTornado for other things, so I ended up
+ using it as the basis to make the tool for creating trackerless torrent
+ files. The modifications were simple. It appears that the original
+ BitTornado CVS tree is off-line (anyways, it was served insecurely),
+ but it looks like
+ <a href="https://github.com/effigies/BitTornado">effigies/BitTornado</a> is
+ similar enough that it could be modified and used. I copied
+ <code>btmakemetafile.py</code> to <code>btmaketrackerless.py</code> and applied the following
+ patch:
+ <pre class="fullwidth"><code>
+ $ diff -u btmakemetafile.py btmaketrackerless.py
+ --- btmakemetafile.py 2004-05-24 12:54:52.000000000 -0700
+ +++ btmaketrackerless.py 2016-10-10 17:13:32.742081000 -0700
+ @@ -23,9 +23,9 @@
+ def prog(amount):
+ print '%.1f%% complete\r' % (amount * 100),
+
+ -if len(argv) < 3:
+ +if len(argv) < 2:
+ a,b = split(argv[0])
+ - print 'Usage: ' + b + ' <trackerurl> <file> [file...] [params...]'
+ + print 'Usage: ' + b + ' <file> [file...] [params...]'
+ print
+ print formatDefinitions(defaults, 80)
+ print_announcelist_details()
+ @@ -33,9 +33,9 @@
+ exit(2)
+
+ try:
+ - config, args = parseargs(argv[1:], defaults, 2, None)
+ - for file in args[1:]:
+ - make_meta_file(file, args[0], config, progress = prog)
+ + config, args = parseargs(argv[1:], defaults, 1, None)
+ + for file in args[0:]:
+ + make_meta_file(file, None, config, progress = prog)
+ except ValueError, e:
+ print 'error: ' + str(e)
+ print 'run with no args for parameter explanations'
+
+ </code></pre>
+ If you notice, the only thing that is done is to drop the first argument,
+ and instead of passing it into <code>make_meta_file</code>, a <code>None</code> is passed
+ instead. This will simply not add trackers to the torrent file.
+<li>I then run the following script to verify the downloaded files, and
+ generate the torrent files:
+ <pre class="fullwidth"><code>
+ $ cat cmp.sh
+ #!/bin/sh -
+ # REL=12.4-RELEASE
+ # RELABV=12.4R
+ # xzcat ~jmg/public_html/FreeBSD-snap/snapshot.idx.xz | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3 wget --no-verbose -c --limit-rate=7000k
+ # XXX - following command 404, manually saved from relase announcement
+ # wget https://www.freebsd.org/releases/$RELABV/announce.asc
+ # wget -c https://download.freebsd.org/ftp/releases/CI-IMAGES/$REL/amd64/Latest/FreeBSD-$REL-amd64-BASIC-CI.raw.xz
+ # wget -c https://download.freebsd.org/ftp/releases/VM-IMAGES/$REL/riscv64/Latest/FreeBSD-$REL-riscv-riscv64.{qcow2,raw,vhd,vmdk}.xz
+
+ grep -h '^ SHA512' announce.asc | sed -e 's/SHA512 (\(.*\)) = \(.*\)/\2 \1/' | sort -k 2 > sha512.from.asc
+
+ while read hash fname; do
+ if [ -e "$fname" ]; then
+ if [ -e "$fname".torrent ]; then
+ echo skipping "$fname"...
+ continue
+ fi
+
+ sigfile=$(grep -l -- "$fname" *.asc | head -n 1)
+ echo checking "$fname", sig in: "$sigfile"
+ #res=`sha512 -q "$fname"`
+ #res=`shasum -a 512 "$fname" | awk '{ print $1 }'`
+ res=$(openssl sha512 < "$fname" | awk '{ print $2 }')
+ echo "File is: $res"
+ if [ x"$res" != x"$hash" ]; then
+ echo missmatch! "$fname"
+ exit 1
+ fi
+ btmaketrackerless.py "$fname" &
+ else
+ echo missing "$fname"
+ exit 1
+ fi
+ done < sha512.from.asc
+
+ </code></pre>
+<li>Once all the torrents have been generated, I then make the magnet
+ links:
+ <pre class="fullwidth"><code>
+ $ cat btmakemagnet.sh
+ #!/bin/sh -
+
+ # metainfo file.: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso.torrent
+ # info hash.....: 06091dabce1296d11d1758ffd071e7109a92934f
+ # file name.....: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso
+ # file size.....: 203161600 (775 * 262144 + 0)
+ # announce url..: udp://tracker.openbittorrent.com:80
+ # btshowmetainfo 20030621 - decode BitTorrent metainfo files
+
+ for i in *.torrent; do
+ btshowmetainfo.py "$i" | awk '
+ $0 ~ "^info hash" { info = $3 }
+ $0 ~ "^file name" { name = $3 }
+ END {
+ print "magnet:?xt=urn:btih:" info "&dn=" name
+ }'
+ done
+ </code></pre>
+<li>I then create the magnet links file, and update the
+ <a href="https://wiki.freebsd.org/Torrents">Torrents</a> wiki page.
+</ol>
diff --git a/layout/blog.j2 b/layout/blog.j2
index ac7b172..38a0798 100644
--- a/layout/blog.j2
+++ b/layout/blog.j2
@@ -9,7 +9,8 @@
<h1 id="{{ slug(resource) }}-title"><a class="no-tufte-underline" href="{{ itemurl(resource) }}">{{ resource.meta.title }}</a></h1>
-<p class="posted">Posted: {% if resource.meta.listable %}{{ resource.meta.created.strftime('%B %e, %Y') }} at {{ resource.meta.time }}{% else %}not yet{% endif %}</p>
+{% if resource.meta.updated %}<p class="posted">Updated: {{ resource.meta.updated.strftime('%B %e, %Y') }} at {{ resource.meta.updated_time }}</p>{% endif %}
+<p class="posted">{% if resource.meta.updated %}Originally {% endif %}Posted: {% if resource.meta.listable %}{{ resource.meta.created.strftime('%B %e, %Y') }} at {{ resource.meta.time }}{% else %}not yet{% endif %}</p>
{% if resource.meta.tags %}
<ul class="tags clear">