blog/content/2022/12/making-freebsd-magnet-links.html

---
title: Making FreeBSD magnet links
description: >
  Making FreeBSD magnet links
created: !!timestamp '2018-07-03'
time: 4:49 PM
updated: !!timestamp '2022-12-07'
updated_time: 12:20 PM
tags:
  - FreeBSD
  - magnet
---

Note: This was originally posted
<a href="/2018/07/making-freebsd-magnet-links.html#making-freebsd-magnet-links-title">here</a>,
but things have been improved since then and so it has been updated.

For the last few years, I've been producing torrents and publishing
magnet links, but there is some special work that I do to make these.
The first few releases, I inserted a bogus tracker into the torrent,
because despite there being plenty of tools out there for producing
trackerless (DHT) torrents, they were all GUI and I never found any that
were command line based.  The other was there was/is no tool for
extracting the info hash and building the magnet link.  There may be
tools now, but I couldn't find any when I started 3 years ago.

The following steps are based upon the recent release of FreeBSD 11.2&#x2011;R,
adjust as necessary.

<ol>
<li>Fetch the FreeBSD release and release announcement into a directory
   (I create a per release directory).
   remove the <code>CHECKSUM.SHA256</code>, <code>CHECKSUM.SHA512</code> and <code>index.html*</code> files.
   <pre class="fullwidth"><code>
   $ REL=12.4-RELEASE
   $ RELABV=12.4R
   $ curl https://www.funkthat.com/~jmg/FreeBSD-snap/snapshot.idx.xz | xzcat | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3  wget --no-verbose -c --limit-rate=7000k
   $ wget https://www.freebsd.org/releases/$RELABV/announce.asc

   </code></pre>
   This step depends upon
   <a href="https://funkthat.com/gitea/jmg/snapaid">snapaid</a> running.
   If it is no longer running for some reason, <code>addinfo.sh</code> could
   be run manually against the release email (not the downloaded
   <code>announce.asc</code> file as the message id is needed) to generate
   snapshot.idx.xz locally, and use that instead.
<li>Verify the GPG key that signed the above files.  This is usually Glen
   Barber's key, but not always.  I have met and verified his fingerprint
   in person,  If you have verified someone's key who has signed Glen's
   key, that is another good way.
<li>Verify the release announcement:
   <pre class="fullwidth"><code>
   $ gpg --verify announce.asc
   Warning: using insecure memory!
   gpg: Signature made Mon Dec  5 17:34:59 2022 PST using RSA key ID 478FE293
   gpg: Good signature from "Glen Barber <gjb@FreeBSD.org>" [unknown]
   gpg:                 aka "Glen Barber <gjb@keybase.io>" [unknown]
   gpg:                 aka "Glen Barber <gjb@glenbarber.us>" [unknown]
   gpg:                 aka "Glen Barber <glen.j.barber@gmail.com>" [unknown]
   gpg: WARNING: This key is not certified with a trusted signature!
   gpg:          There is no indication that the signature belongs to the owner.
   Primary key fingerprint: 78B3 42BA 26C7 B2AC 681E  A7BE 524F 0C37 A0B9 46A3
        Subkey fingerprint: 8D12 403C 2E6C AB08 6CF6  4DA3 0314 58A5 478F E293
   </code></pre>
<li>In the past I have used BitTornado for other things, so I ended up
   using it as the basis to make the tool for creating trackerless torrent
   files.  The modifications were simple.  It appears that the original
   BitTornado CVS tree is off-line (anyways, it was served insecurely),
   but it looks like
   <a href="https://github.com/effigies/BitTornado">effigies/BitTornado</a> is
   similar enough that it could be modified and used.  I copied
   <code>btmakemetafile.py</code> to <code>btmaketrackerless.py</code> and applied the following
   patch:
   <pre class="fullwidth"><code>
   $ diff -u btmakemetafile.py btmaketrackerless.py 
   --- btmakemetafile.py   2004-05-24 12:54:52.000000000 -0700
   +++ btmaketrackerless.py        2016-10-10 17:13:32.742081000 -0700
   @@ -23,9 +23,9 @@
    def prog(amount):
        print '%.1f%% complete\r' % (amount * 100),

   -if len(argv) &lt; 3:
   +if len(argv) &lt; 2:
        a,b = split(argv[0])
   -    print 'Usage: ' + b + ' &lt;trackerurl&gt; &lt;file&gt; [file...] [params...]'
   +    print 'Usage: ' + b + ' &lt;file&gt; [file...] [params...]'
        print
        print formatDefinitions(defaults, 80)
        print_announcelist_details()
   @@ -33,9 +33,9 @@
        exit(2)

    try:
   -    config, args = parseargs(argv[1:], defaults, 2, None)
   -    for file in args[1:]:
   -        make_meta_file(file, args[0], config, progress = prog)
   +    config, args = parseargs(argv[1:], defaults, 1, None)
   +    for file in args[0:]:
   +        make_meta_file(file, None, config, progress = prog)
    except ValueError, e:
        print 'error: ' + str(e)
        print 'run with no args for parameter explanations'

   </code></pre>
   If you notice, the only thing that is done is to drop the first argument,
   and instead of passing it into <code>make_meta_file</code>, a <code>None</code> is passed
   instead.  This will simply not add trackers to the torrent file.
<li>I then run the following script to verify the downloaded files, and
   generate the torrent files:
   <pre class="fullwidth"><code>
   $ cat cmp.sh
   #!/bin/sh -
   # REL=12.4-RELEASE
   # RELABV=12.4R
   # xzcat ~jmg/public_html/FreeBSD-snap/snapshot.idx.xz | awk ' $2 == "'"$REL"'" { print $9 }' | xargs -L 1 -P 3  wget --no-verbose -c --limit-rate=7000k
   # XXX - following command 404, manually saved from relase announcement
   # wget https://www.freebsd.org/releases/$RELABV/announce.asc
   # wget -c https://download.freebsd.org/ftp/releases/CI-IMAGES/$REL/amd64/Latest/FreeBSD-$REL-amd64-BASIC-CI.raw.xz
   # wget -c https://download.freebsd.org/ftp/releases/VM-IMAGES/$REL/riscv64/Latest/FreeBSD-$REL-riscv-riscv64.{qcow2,raw,vhd,vmdk}.xz

   grep -h '^   SHA512' announce.asc | sed -e 's/SHA512 (\(.*\)) = \(.*\)/\2 \1/' | sort -k 2 &gt; sha512.from.asc

   while read hash fname; do
           if [ -e "$fname" ]; then
                   if [ -e "$fname".torrent ]; then
                           echo skipping "$fname"...
                           continue
                   fi

                   sigfile=$(grep -l -- "$fname" *.asc | head -n 1)
                   echo checking "$fname", sig in: "$sigfile"
                   #res=`sha512 -q "$fname"`
                   #res=`shasum -a 512 "$fname" | awk '{ print $1 }'`
                   res=$(openssl sha512  &lt; "$fname" | awk '{ print $2 }')
                   echo "File is: $res"
                   if [ x"$res" != x"$hash" ]; then
                           echo missmatch!  "$fname"
                           exit 1
                   fi
                   btmaketrackerless.py "$fname" &
           else
                   echo missing "$fname"
                   exit 1
           fi
   done &lt; sha512.from.asc

   </code></pre>
<li>Once all the torrents have been generated, I then make the magnet
   links:
   <pre class="fullwidth"><code>
   $ cat btmakemagnet.sh
   #!/bin/sh -

   # metainfo file.: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso.torrent
   # info hash.....: 06091dabce1296d11d1758ffd071e7109a92934f
   # file name.....: FreeBSD-10.3-RELEASE-sparc64-bootonly.iso
   # file size.....: 203161600 (775 * 262144 + 0)
   # announce url..: udp://tracker.openbittorrent.com:80
   # btshowmetainfo 20030621 - decode BitTorrent metainfo files

   for i in *.torrent; do
           btshowmetainfo.py "$i" | awk '
   $0 ~ "^info hash" { info = $3 }
   $0 ~ "^file name" { name = $3 }
   END {
           print "magnet:?xt=urn:btih:" info "&dn=" name
   }'
   done
   </code></pre>
<li>I then create the magnet links file, and update the
   <a href="https://wiki.freebsd.org/Torrents">Torrents</a> wiki page.
</ol>