Jekyll2020-08-09T04:30:03-05:00https://blog.night-shade.org.uk//feed.xmlA travelling tinkerThings I've fixed, places I've beenTeams, MFA and AWS root accounts2018-06-17T00:00:00-05:002018-06-17T00:00:00-05:00https://blog.night-shade.org.uk//2018/05/aws-root-account-paper-mfa-backup<p>Enabling two factor authentication is one of the best ways to keep your AWS and other accounts secure. There are a lots of ways to do this and one of the popular tools for this is Google Authenticator, which shows a 6 number rolling code that changes over time. <a href="https://en.wikipedia.org/wiki/Google_Authenticator">Behind the scenes</a> this code is generated from a shared secret between your phone and the service you are logging into, hashed with the current time.</p>
<p>The root account of an AWS account isn’t like a normal account. It is rarely used after inital setup, needs to be kept secure but accessible in a crisis for a team of people. This makes MFA configuration more complex especially as team members come and go.</p>
<p>An MFA workflow a former team used successfully is:</p>
<ul>
<li>Generate and print the MFA QR code</li>
<li>Scan and activate the QR code using a cheap android mobile phone</li>
<li>Securely store the mobile phone, maybe turned off to save power</li>
<li>Security store the MFA QR code printed out in an envelope separately from the android phone</li>
</ul>
<p>You could also use a shared Yubikey to store the MFA code rather than a mobile phone by using <a href="https://www.yubico.com/products/services-software/download/yubico-authenticator/">Yubico Authenticator
</a>.</p>
<p>This solves in a fairly secure way the following:</p>
<ul>
<li>Continuing access to an AWS root account when personal change</li>
<li>Quick access to the root account if needed</li>
<li>Retain access to MFA when technology fails</li>
<li>Separation of password and MFA tokens</li>
</ul>Tim FletcherEnabling two factor authentication is one of the best ways to keep your AWS and other accounts secure. There are a lots of ways to do this and one of the popular tools for this is Google Authenticator, which shows a 6 number rolling code that changes over time. Behind the scenes this code is generated from a shared secret between your phone and the service you are logging into, hashed with the current time.Moving my blog to Jekyll2018-05-12T15:49:00-05:002018-05-12T15:49:00-05:00https://blog.night-shade.org.uk//2018/05/moving-to-jekyll<p>I’ve been using markdown more and more in my day job to write documentation and so I am now in the process of moving from Wordpress to Jekyll</p>
<p>I’m also bored of dealing with Wordpress and MySQL running out of RAM….</p>Tim FletcherI’ve been using markdown more and more in my day job to write documentation and so I am now in the process of moving from Wordpress to JekyllYubikey hardware and SSH keys for macOS2016-10-14T06:49:29-05:002016-10-14T06:49:29-05:00https://blog.night-shade.org.uk//2016/10/yubikey-hardware-and-ssh-keys-for-macos<p>Setting up Yubikey hardware backed SSH keys for Linux was a total pain, but setting it up on macOS was actually very simple. Note this is for command line, I’ve not looked at setting this up for GUI applications.</p>
<p>I am assuming that you already have a working GPG key on your Yubiky and want to set it up for SSH login.</p>
<p>First you will need to install <a href="https://gpgtools.org/">GPGTools</a></p>
<p>Next you will need to set up the gpg-agent config file, add the following lines to the file <code class="language-plaintext highlighter-rouge">$HOME/.gnupg/gpg-agent.conf</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>enable-ssh-support
use-standard-socket
write-env-file
</code></pre></div></div>
<p>The final change you need to make is in $HOME/.bash_profile to add these lines:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>unset SSH_AUTH_SOCK
export SSH_AUTH_SOCK=$HOME/.gnupg/S.gpg-agent.ssh
</code></pre></div></div>Tim FletcherSetting up Yubikey hardware backed SSH keys for Linux was a total pain, but setting it up on macOS was actually very simple. Note this is for command line, I’ve not looked at setting this up for GUI applications.Three weeks and a few thousand miles on the Indian railway2016-02-07T11:53:50-06:002016-02-07T11:53:50-06:00https://blog.night-shade.org.uk//2016/02/three-weeks-and-a-few-thousand-miles-on-the-indian-railway<p>I’ve been in India since the 19th of January, and I’ve now travelled a few thousand kilometres on the Indian Railways. Starting in Chennai (Madras) I heading down to Thiruvananthapuram (aka <span class="st">Trivandrum) the capital of Kerala state in the far south of India. I travelled up the west coast of India to Goa, Mumbai (Bombay) and now I’m writing this from Jodhpur.</span></p>
<p>I have mostly travelled on the long distance trains in either AC1 or AC2 sleeper coaches which I know isn’t how most of the millions of Indians travel by train. I did travel into and out of the centre of Mumbai on the famously crowded suburban trains.</p>
<p><img src="https://blog.night-shade.org.uk//wp-content/uploads/2016/02/thumb_IMG_9166_1024.jpg" alt="Just like the underground, only with fewer doors." /></p>
<p>The people I have met on the trains have been without exception friendly, hospitable and willing to help. I have shared an evening meal with Indian families on the train between Chennai and Trivandrumand with each family making sure that I had tried their own speciality.</p>
<p>I have since discovered that you can <a href="http://www.travelkhana.com/">order food</a> for delivery to your seat on the train as it stops as intermediate stations. I used this going between Mumbai and Jodhpur, getting a meal delivered to my seat for 155/- Rs or about £1.55.</p>
<p><img src="https://blog.night-shade.org.uk//wp-content/uploads/2016/02/thumb_IMG_9200_1024.jpg" alt="Curry delivered to my seat" /></p>
<p>I have been on a train that was held up by monkeys, an excuse that Northern Rail have not yet tried! You can see one of the monkeys in question climbing on the cables in the picture.</p>
<p><img src="https://blog.night-shade.org.uk//wp-content/uploads/2016/02/thumb_IMG_9112_1024.jpg" alt="Monkeys on the signal cables" /></p>
<p>I have seen a different side to India for example trundling through the backwaters of Kerala.</p>
<p><img src="https://blog.night-shade.org.uk//wp-content/uploads/2016/02/thumb_IMG_8977_1024.jpg" alt="Washing drying in the jungle" /></p>Tim FletcherI’ve been in India since the 19th of January, and I’ve now travelled a few thousand kilometres on the Indian Railways. Starting in Chennai (Madras) I heading down to Thiruvananthapuram (aka Trivandrum) the capital of Kerala state in the far south of India. I travelled up the west coast of India to Goa, Mumbai (Bombay) and now I’m writing this from Jodhpur.Looking forward to 20162016-01-03T08:05:39-06:002016-01-03T08:05:39-06:00https://blog.night-shade.org.uk//2016/01/looking-forward-to-2016<p>This isn’t how a positive posts normally starts, but both Serra and I are now unemployed and looking forward to a whole year of not having a job!</p>
<p>We are in the last stages of getting ready to go travelling, we have all our visas and vaccinations, the front room looks like a bomb site and the idea we only have 2 weeks left in the UK has really hit home!</p>
<p>We are about to embark on a tour of the UK to say goodbye to friends and family. We will be in Warwickshire Monday and Tuesday this week, then London next weekend (7-11th of Jan) and finally Plymouth for a few days afterwards. If anyone wants to catch us for a coffee, beer or other beverage drop me a line on social media.</p>
<p>Travel wise we are in India January to March, then China for March and fly to Japan for April. We will be travelling downwards through South East Asia for May and June towards Australia. We are planning to be back in Europe for 4 weeks from mid August before heading off to South America for the rest of the year and the start of 2017. We would love to catch up with friends and family as we travel.</p>
<p> </p>
<p> </p>Tim FletcherThis isn’t how a positive posts normally starts, but both Serra and I are now unemployed and looking forward to a whole year of not having a job!Why we are quitting our jobs and setting off to see the world2015-07-19T08:37:45-05:002015-07-19T08:37:45-05:00https://blog.night-shade.org.uk//2015/07/why-we-are-quitting-our-jobs-and-setting-off-to-see-the-world<p>Some of you might know that my wife and I are planning to quit our jobs at the end of this year and travel the world together.</p>
<p>Sadly 2 1/2 years ago my grandfather died aged 89, and during the service his daughter (my aunt) said in her eulogy “His wife died young while he was still working and he always regretted not spending time with her”. While I can’t fix that, I can try to make sure that neither Serra or I have that regret.</p>
<p>This time <a href="https://goo.gl/photos/ZHoLC44UtRKrN4y1A">10 years ago</a> I was in Eastern Europe travelling with friends on the railways and loving the experience. I went on to travel to Northern Europe regularly in the next few years, meeting online friends “In Real Life”.</p>
<p>5 years ago today I was getting married to Serra, surrounded by friends and family. We are lucky enough (we know we are lucky) to be able leave our jobs and set off to experience the world together. We both feel the need to evaluate what we in terms of employment and lifestyle for the next few decades (hopefully) together.</p>
<p>I have built the <a href="https://www.google.com/maps/d/edit?mid=zRqFVbP9-ia0.khved-wZKp_Q&usp=sharing">outline of the route</a> where we want to go, so if anyone wants to meet as we travel get in touch.</p>Tim FletcherSome of you might know that my wife and I are planning to quit our jobs at the end of this year and travel the world together.Why Does Dropbox Add a Unique ID to Every Photo?2015-06-04T18:51:51-05:002015-06-04T18:51:51-05:00https://blog.night-shade.org.uk//2015/06/why-does-dropbox-add-a-unique-id-to-every-photo<p>Following on from my last post about Dropbox changing my photos, I noticed a new exif field of “Image Unique ID” embedded by Dropbox in the image.</p>
<p>This ID would allow Dropbox to track unique files across their storage estate to avoid duplication. Equally it could used to track the original file and who uploaded it from a cropped version posted online, especially if law enforcement turned up with legal papers and demanded access.</p>
<p>Think about leaked documents or protest photos, yes it’s good practise to strip the meta data out but not everyone does.</p>
<p>This again comes back to what Dropbox and it’s camera upload feature is doing and is it documented anywhere?</p>
<p>Note Google Photos does not embedded any tracking data in the exif of the image I tested by uploading and downloading it.</p>
<h2 id="the-hash">The Hash</h2>
<p>The hash for IMG_7082 is 8af323e74def610b0000000000000000 which looks like a 128bit hash but with only the first 64 bits populated. I’ve tried a number of hash tools on various parts of the original but they don’t match the unique ID. I have tested just the pure image data from original and Dropbox modified images.</p>
<p>Reverse engineering the hash function isn’t the real issue here, the real question is why has this ID been added?</p>Tim FletcherFollowing on from my last post about Dropbox changing my photos, I noticed a new exif field of “Image Unique ID” embedded by Dropbox in the image.Dropbox iPhone Camera Upload Changes Photos2015-06-03T18:04:08-05:002015-06-03T18:04:08-05:00https://blog.night-shade.org.uk//2015/06/dropbox-iphone-camera-upload-changes-photos<p>When Google announced their new <a href="https://photos.google.com">Photos</a> tools I decided to give it a go and see what Google’s machine learning could extract from my 83,292 photos stretching back 15 years. I’m sure you know that Google are offering “unlimited” and “free” storage for photos so long as you allow them to optimize your photos. I’m happy with the trade-off in quality as I already manage an archive of full resolution (or so I thought) photos via <a href="http://f-spot.org/">f-spot</a> and have backup arrangements for it.</p>
<h2 id="dropbox-camera-upload">Dropbox Camera Upload</h2>
<p>I have used the Dropbox Camera Upload feature for about 18 months to get photos off my iPhone and on to my various other devices and offsite backup server. Dropbox <a href="https://blogs.dropbox.com/dropbox/2012/06/your-photos-simplified-part-iii/">state</a> that “When you open the app, photos and videos from your iPhone or iPad are saved to Dropbox at their original size and quality in a private Camera Uploads folder.”</p>
<p>This statement hides the fact, that the Dropbox app re-compresses your photos before it uploads them. I found this out when I used the desktop backup client to seed Google Photos from my Dropbox camera folder, before activating the apps on my iPhone and iPad.</p>
<p>Google checksum all photos before uploading to avoid duplication. When I enabled the Google Photos app on my iOS devices to upload directly from the iOS camera roll, the app started to upload all my photos again. This led to duplicated photos and a few gigs of wasted upload bandwidth. I wanted to understand why this happened and adapt my photo work flow to avoid it happening again.</p>
<h2 id="image-checksums">Image checksums</h2>
<p>First of all I extracted a single photo IMG_7082 taken that day directly from my iPhone over USB. I copied the file from the DCIM folder on the phone, gaving me a 2.8MB file as my “master copy”. Checking my Dropbox “Camera Uploads” folder I found the same photo as expected had been <a href="https://www.dropbox.com/en/help/4208">renamed by Dropbox</a> but unexpectedly had a different checksum and was over 1 megabyte smaller, the plot thickens!</p>
<p>The obvious next question was what is changing the file, so I extracted the same image file via email (sent as full resolution), iCloud and Photos on my MacBook each time it was the same size with a matching sha1 checksum. Uploading the master file to the free tier of Google Photos and then extracting it via Google Drive or the web UI did change the file but Google are upfront about that.</p>
<h2 id="the-proof">The Proof</h2>
<p>I have created a github repo with all the photos I used in testing if you want to have a look at them yourself it’s here: <a href="https://github.com/TimJDFletcher/IMG_7082">https://github.com/TimJDFletcher/IMG_7082</a></p>
<h2 id="quality-change">Quality Change</h2>
<p>I had a quick go at reproducing the same change in size of the image using GIMP and changing the JPEG compression level. I found that at 85% the file size was very close to the file size the both Google Photos and Dropbox produced. This is pretty crude test and is not to say this is the only compression that Google and Dropbox do.</p>
<h2 id="lessons-learned">Lessons Learned</h2>
<p>The main lesson for me is that I should confirm how applications I rely on to move data work as advertised. I do understand why Dropbox re-compress photos as it gives a large saving in storage and bandwidth, I wish they were as upfront about this as Google are.</p>
<p>Google says they will optimize your photos, if you don’t like this then you can pay money to store the originals. Dropbox on the other hand say “Don’t worry about losing those once-in-a-lifetime shots, no matter what happens to your iPhone.”</p>
<h2 id="fixing-the-duplicates">Fixing the duplicates</h2>
<p>Fixing the duplicates was fairly simple in the end, I just got a list of files uploaded from my iPhone and then deleted them from Google Drive using <a href="https://github.com/astrada/google-drive-ocamlfuse">google-drive-ocamlfuse</a> and a bit of shell script.</p>Tim FletcherWhen Google announced their new Photos tools I decided to give it a go and see what Google’s machine learning could extract from my 83,292 photos stretching back 15 years. I’m sure you know that Google are offering “unlimited” and “free” storage for photos so long as you allow them to optimize your photos. I’m happy with the trade-off in quality as I already manage an archive of full resolution (or so I thought) photos via f-spot and have backup arrangements for it.Fixing LDAP error 53 “Server is unwilling to perform”2015-05-31T17:36:14-05:002015-05-31T17:36:14-05:00https://blog.night-shade.org.uk//2015/05/fixing-ldap-error-53-server-is-unwilling-to-perform<p>Work has a pair of OpenLDAP servers which are in a standard master / slave synchronized setup. While preparing for some updates I checked that the LDAP servers where syncing correctly and discovered that the slave hadn’t updated in over 6 months!</p>
<p>On the slave server /var/log/syslog contained the following errors:</p>
<pre>slapd[PID]: do_syncrep2: rid=XXX LDAP_RES_SEARCH_RESULT (53) Server is unwilling to perform
slapd[PID]: do_syncrep2: rid=XXX (53) Server is unwilling to perform</pre>
<p>Working through the <a href="https://help.ubuntu.com/14.04/serverguide/openldap-server.html">Ubuntu server guide</a> used to set up the pair of servers in the first place didn’t shed any light on the problem. A fresh Ubuntu 14.04 server in GCE showed the same problem, so at least I know the problem is on the master server.</p>
<p>I finally got a clue from <a href="http://www.zytrax.com/books/ldap/ch12/">chapter 12</a> of “<a href="http://www.zytrax.com/books/ldap/">LDAP for Rocket Scientists</a>{.t-db}“, which suggested that the master server had “no global superior knowledge”. This was enough to make me test removing the accesslog databases, which track LDAP transactions and allow slave servers to sync changes from the master.</p>
<p>In the end it was a simple as removing the databases from /var/lib/ldap/accesslog and letting slapd rebuild them after a restart. Note depending on the config in slapd this might be in a different directory, check the setting for olcDbDirectory with this command:</p>
<pre>sudo slapcat -b cn=config -a "(|(cn=config)(olcDatabase={2}hdb))"</pre>Tim FletcherWork has a pair of OpenLDAP servers which are in a standard master / slave synchronized setup. While preparing for some updates I checked that the LDAP servers where syncing correctly and discovered that the slave hadn’t updated in over 6 months!Exchange Cumulative Update 8 and SSL certificates2015-05-18T10:05:25-05:002015-05-18T10:05:25-05:00https://blog.night-shade.org.uk//2015/05/exchange-cumulative-update-8-and-ssl-certs<p>This weekend while I was patching and rebooting KVM systems for Venom I took the opportunity to apply Microsoft’s latest <a href="https://support.microsoft.com/en-gb/kb/3030080">Exchange Cumulative Update 8</a> work’s Exchange server.</p>
<p>I ran the pre-upgrade checks and they picked up that my user wasn’t in the correct AD groups for scheme updates, once that was fixed the upgrade started without problems.</p>
<p>When the upgrade got to Section 10 – Mailbox role: Transport Services the upgrade failed because a certificate had expired. Fine just install an updated certificate but all of the Exchange management tools have been uninstalled so you can’t get to the certificate.</p>
<p>When you rerun the installer it detects the failed install and tries to resume and fails at the same place, in the end I had to move the clock back two days on the server to get the management tools to install so that the certificate could be replaced.</p>
<p>I’ve reviewed the Microsoft documentation and I can’t see any reference to this problem and it wasn’t detected in the pre-upgrade checks. I’m sure there is some powershell magic that could fix this but at 1am on a Monday morning I wasn’t all that interested in finding out!</p>Tim FletcherThis weekend while I was patching and rebooting KVM systems for Venom I took the opportunity to apply Microsoft’s latest Exchange Cumulative Update 8 work’s Exchange server.