Private git server with gitea

PURPOSE

With Microsoft acquiring GitHub (for $7.5 BILLION), I now have incentive to host my own Git repositories. For the longest time I had thought that GitHub was an Open Source project, but then I was stymied when I tried to find a way to host my own GitHub server (it seems you need to be an Enterprise to host a proper private GitHub).

Fast forward a couple of years, and Vivek Gite’s *nixcraft blog post on the subject was linked to me via email. The purpose of this article is to document my efforts to install Gitea. I had tried to install it at https://eldon.me/git/, but that would require retooling the other web application at https://eldon.me/ (WordPress). The folks on IRC (#nginx@freenode) said to make a subdomain, rather than a subdirectory. After being reminded that it’s easy to add CNAMEs to my DNS records, I now have https://git.eldon.me.

Why do this? I’d like to start using git more for my personal projects. I’d rather not store sensitive materials anywhere but something I fully control. Also, being able to link folks to my own repository rather Debian’s (or whoever’s) paste bin when I’m having issues is quite attractive to me.

PREREQUISITES

  • A Linux server (mine is a ChunkHost chunk running Debian 9.4 [stretch])
  • nginx installed (with optional SSL/TLS certificates [HIGHLY RECOMMENDED])
  • A database engine (I already have MariaDB [10.1.26-MariaDB] installed). If in doubt, go with Gitea’s built-in SQLite3 database
  • SSH service enabled on the target host (at an optional nonstandard port)

PROCEDURE

The following instructions use non-root user and host in the commands. Change accordingly.

  1. Log into the target host via SSH (user will be assumed to *NOT* be root):
    ssh host -l user
  2. Make a staging directory and change to it:
    mkdir -p ~/src/gitea && cd ~/src/gitea
  3. Install prerequisite packages git, golang (from stretch-backports), wgetand zip:
    sudo apt install git wget zip
    sudo apt -t stretch-backports install golang
  4. Add a new user for Gitea
    sudo adduser --disabled-login --gecos 'Gitea' git
  5. Change this new user git:
    sudo -u git -i
  6. Get the latest version of gitea (currently v1.4.2)
    mkdir -p bin
    wget -O bin/gitea https://dl.gitea.io/gitea/1.4.2/gitea-1.4.2-linux-amd64
    chmod +x bin/gitea
  7. Exit the git user shell
    exit
  8. Create systemd service file /etc/systemd/system/gitea.service for Gitea:
          [Unit]
          Description=Gitea
          After=syslog.target
          After=network.target
          After=mariadb.service mysqld.service postgresql.service memcached.service redis.service
        
          [Service]
          # Modify these two values and uncomment them if you have
          # repos with lots of files and get an HTTP error 500 because
          # of that
          ###
          #LimitMEMLOCK=infinity
          #LimitNOFILE=65535
          Type=simple
          User=git
          Group=git
          WorkingDirectory=/home/git
          ExecStart=/home/git/bin/gitea web
          Restart=always
          Environment=USER=git HOME=/home/git
        
          [Install]
          WantedBy=multi-user.target
  9. Start Gitea:
          sudo systemctl enable gitea
          sudo systemctl start gitea
          
  10. Create an nginx site configuration file /etc/nginx/sites-available/git.host:
    server {                                                                                                                                                                                      
        listen 80;                                                                                                                                                                            
        listen [::]:80;                                                                                                                                                                       
        server_name git.host;                                                                                                                                                             
        return 301 https://git.host;                                                                                                                                          
                                                                                                                                                                                                  
        # Redirect non-https traffic to https                                                                                                                                                     
        # if ( != https) {                                                                                                                                                               
        #     return 301 https://;                                                                                                                                               
        # } # managed by Certbot                                                                                                                                                                  
                                                                                                                                                                                                  
    }                                                                                                                                                                                             
                                                                                                                                                                                                  
    server {                                                                                                                                                                                      
        listen 443 ssl;                                                                                                                                                                           
        server_name git.host;
        ssl_certificate /etc/letsencrypt/live/host/fullchain.pem; # managed by Certbot                                                                                                     
        ssl_certificate_key /etc/letsencrypt/live/host/privkey.pem; # managed by Certbot                                                                                                   
                                                                                                                                                                                                  
        root /var/www/git.host/;                                                                                                                                                              
        location / {                                                                                                                                                                              
                client_max_body_size 364M;                                                                                                                                                        
                proxy_set_header Host ;                                                                                                                                                      
                proxy_set_header X-Real-IP ;                                                                                                                                          
                proxy_pass http://localhost:3000;                                                                                                                                                 
                proxy_connect_timeout 600;                                                                                                                                                        
                proxy_send_timeout 600
        }
    }
    
  11. Enable the new git.host:
    ln -s /etc/nginx/sites-{available,enabled}/git.host
  12. Restart nginx
    sudo systemctl restart nginx
  13. Enter MariaDB client shell (admin account)
    mysql -u root -p'password'
  14. Add gitea database
    CREATE DATABASE gitea;
  15. Add gitea user
    CREATE USER 'gitea' IDENTIFIED BY 'new_password';
  16. Grant privileges to gitea user
    GRANT ALL PRIVILEGES ON gitea.* TO 'gitea'@localhost IDENTIFIED BY 'new_password';
  17. Exit the MariaDB mysql client shell
    exit
  18. Now, you’re ready to configure Gitea at https://git.host/!

(trey’s take)On sharing a directory with Windows…

Market forces have conspired into forcing me to have a bare-metal Windows install (a Virtualbox Virtual Machine can no longer cut it). I still would rather work solely in Debian, or some form of Linux distribution (distro!), but this presents me with an opportunity to grow my skills in ways I did not anticipate. First, the scenario:

  1. I use the venerable KeePassX to manage my passwords. It is not to be confused with KeePass. Both programs have similar functions, and even read the same database format (KeePass 1.x), but the latest (and currently maintained) KeePass version depends on .NET, so on Linux that means Mono. Last I checked (which is admittedly a long time ago), the Linux/Mono port looked *terrible*. Luckily, KeePassX has a port for Windows, so I can keep the same look and feel regardless of whether I’m booted into Windows or Linux. FULL DISCLOSURE: KeePassX 0.4.3 has been out for some time, and it appears the development on the next version of KeePassX has slowed to a crawl or is nonexistent.
  2. No matter which computer I’m using, I want to be able to access the password database. In Linux, I just set up a port forward on my WAN router that points to the SSH port on my Debian workstation, and on my satellite devices I use SSHFS to mount the keepass directory. This “tricks” KeePassX in thinking the password database is local to the satellite machine, any changes are immediately available on the main workstation and satellite machines, and there is no reconciling disparate databases. As part of the (manual) SSHFS mount command, I make a local copy on the satellite so the password is available if my central server is not. Note, if my primary home workstation becomes unavailable, and I need to modify the database in any way, I will need to manually merge my KeePassX databases.

My previous architecture is described above. Adding a dual-boot Windows 7 installation to the mix gives me a number of challenges:

  1. There is the problem of sharing the database between both OSes (Windows 7 and Debian), such that both versions of KeePassX operate without having to redirect KeePassX to a different location (C:\Users\trey\keepass in Windows, and /home/trey/keepass in Debian).
  2. Making the database available via SSHFS will pose a challenge on the Windows side.
  3. Making backups of the database may be difficult from the Windows side

My solutions to the above:

  1. Sharing the database between the two systems is relatively straightforward. This is where Linux plays the glue system, and makes up for the inadequacies of others. Windows 7 will store the master password database here: C:\Users\trey\keepass\. I can mount the C: drive in Linux (the ntfs-3g filesystem driver is quite mature), and then bind mount the Windows directory to /home/trey/keepass/. Here are the relevant /etc/fstab entries:
    /dev/sda2                       /windows           ntfs    defaults,uid=1000 0 0
    /windows/Users/trey/keepass     /home/trey/keepass none    bind              0 0
    

    If I’m booted into Linux, everything is as it was. I don’t anticipate that the database actually residing on an NTFS volume to be a concern, but usage may dictate otherwise.

  2. Making the directory available from the Debian side is already done, and works as expected. Doing so from Windows is a bit more difficult. I’ll need to install OpenSSH in Cygwin, which is done, but it’s not configured. I’ll first need to expose C:\Users\trey\keepass in the Cygwin filesystem tree. I’ll also need to copy the various SSH keys into the Cygwin environment, so my satellite devices don’t know (or care) when they mount the SSHFS volume. I’m currently writing this from Debian; I’ll need to reboot into Windows and continue this post later.
  3. I have no idea how my general rsnapshot backup will work if Windows is booted. I’ll have to think of that later, but assuming the keepass directory is properly exposed in Windows/Cygwin, it should be OK.

On to the Windows side…

Increase random entropy pool in Debian sid

Hopefully this will be a short post. I saw some folks in IRC (one of the many #debian channels I’m connected to) chatting about /dev/urandom and /dev/random, and increasing its available entropy pool. This entropy pool is where all the random numbers generated by a Linux system come from. The higher the entropy pool value, the more truly random the pseudorandom number generator (PRNG) like /dev/random and /dev/urandom can be. This has a specific impact on computer cryptography: if your random number pool is low on entropy, its sequence of random numbers can be guessed relatively easily. I found this Wikipedia article which briefly describes the technology on various operating systems.

The operating system file (well, in the /proc pseudo-filesystem) which displays how much entropy my system currently has is /proc/sys/kernel/random/entropy_avail. It will change over time; to watch it change I used this command:

watch -n 1 cat /proc/sys/kernel/random/entropy_avail

This showed my entropy fluctuating between 100 and 200, which is pretty low and not very useful (or secure). I did some research to try and discover a way to increase this entropy pool. Probably the best option is a hardware random number generator (HRNG), maybe sometimes called a true random number generator (TRNG). These cost money, money I don’t have for spending. I found randomsound, but running it did not appear to affect my entropy one way or the other (probably because on my home machine I don’t have a mic). I found this blog post, but it initially suggests a questionable method to increase entropy. Its update, quietly hidden at the top of the post, gives the solution I came upon.

The solution was to use haveged. This uses nondeterministic optimizations available in modern CPU hardware as its random source. When I ran it with the default options, my entropy pool shot up to between 1024 and 4096. Much improved. In a post further on down on Chris’s blog, someone suggested using the /proc/sys/kernel/random/poolsize as the lower threshold, with the -w option. Debian provides an /etc/default/haveged file where you can place these options:

DAEMON_ARGS="-w $(cat /proc/sys/kernel/random/poolsize)"

Currently, poolsize is set to 4096. Should a new kernel from the Debian team set this pool to be different, haveged will automatically be set to whatever value it is. I have successfully set this on my main workstation machines at work and at home. I will set this on my laptop and my VPS systems, and see how it goes.

UPDATE: All but one of my VPS systems was able to use haveged. The one outlier was because it’s on an OpenVZ VM system, and I don’t have access to those particular parts of the kernel (even as root). I have relegated that VPS to being just a toy, since I can’t really use it for much else. I will probably cancel my subscription to that altogether. We’ll have to see about that.

Adding Windows 7 to my PXE boot server…

I would have thought this would be difficult, but I found this Install Windows 7 over PXE from Linux without WAIK guide. I will document what I had to do differently (since that guide is designed for CentOS 5.3).

The first major change I needed to make was in the /etc/init.d/binl script; it called some things (e.g. /etc/sysconfig/network) which just doesn’t exist in Debian. I had to rip out all of the CentOS/Red Hat bits, and replace them with Debian ones. I also had to edit the binlsrv2.py file, since it had the tftp root path hardcoded in it.

Since tftpd-hpa was already set up (see my previous posts on the PXE boot subject), I just needed to add a remap file to /srv/tftp/windows/remap. This will cause tftpd-hpa to remap backslashes (‘\’) to slashes (‘/’). Setting up samba was pretty simple. I created the [REMINST] share as instructed, and mounted the Windows 7 x64 ISO to that share.

Importing the default boot files was straightforward, just used the wimextract tool found in the win7pxelinux.tgz distributed by the author of the ultimatedeployment.org site. I placed all of the executable files in /usr/local/bin, so I wouldn’t have to adjust my $PATH variable. I ran into an issue generating the winpe.wim Windows image (kept giving me an error that //windows/system32 couldn’t be found). To work around this problem, I added a “mkdir //windows/system32” command to the actionfile.txt, and it worked. Well, at least it didn’t give me an error.

The next step was to run a Perl script called bcdedit.pl. For this I had to install libhivex-bin, so I could run hivexsh (executed by bcdedit.pl). There was a problem, though. The hivexsh script generated a large number of errors. As far as I could figure out, hivexsh is slightly different between the Red Hat way and Debian way. In Debian, the -w option does not take an argument, whereas in Red Hat/CentOS the argument is the hive (Windows registry) file. Correcting this in bcdedit.pl was just a matter of moving the $bcd variable to the end of the command, rather than directly after the -w option.

Now was time to test it out. It looks like it launches, but it can’t seem to pull the files it needs from TFTP. It looks like it’s trying, just can’t find the files it wants:

…after some trial and error I notice that the WinPE client can’t find ‘hiberfil.sys’, a file created by Windows 7 when it hibernates. Thus the installation halts. Just touching the file isn’t enough. I can’t even make one in my Windows 7 virtual machine, since the guest firmware doesn’t support hibernation (not that it really needs to). So I’m stuck trying to figure out where I can grab that file from. I think it’s in the BCD I generated, but I don’t know where. Using the wimextract command from the toolset requires you to know where it is in the path. wimlib, which comes with an implementation of Windows’ imagex. I will have to start a new blog post for that, since I’ll have to start from scratch.