Computers

A tool for file recovery cleanup

If you’ve ever run file recovery tools on a disk, you know that you can end up with multiple copies of recovered files. Well, I made a little script that can help reduce the number of duplicates for you to clean up.

https://github.com/eltopo1971/file-duplicate-nuker

fileDuplicateNuker takes a directory as an argument, then recursively goes through that directory and takes a hash signature from the files in it. When it encounters a file with the same hash signature, it deletes the file.

Does this take care of all the duplicates? Oh heavens no. That’s a feature, not a bug — call it erring on the side of safety. The script has no idea what kind of file it’s dealing with. All it does is take a hash signature and base the decision of whether to delete the file on that. If there is so much as one byte of difference in the file it’s examining, it’s counted as a unique file and not deleted.

That being said, from my testing it does delete a good number of files, and when you have thousands of files to wade through, any little bit helps.

Running a Windows virtual machine on Linux using an Existing Windows Installation

I’ve been a Linux user for years. At home I’ve kept a file server which runs off Linux, but that was “classic Linux” — without a graphical user interface, using the command line, and doing things the “hard way”. My main desktop PC was still running Windows, and had to because the one game I play regularly, Fortnite, was not available on Linux.

Well, someone recommended that I check out GeFORCE NOW, which is a virtualized environment you can use within Linux to play a large number of games in the cloud. I’ve always been skeptical about that type of play, largely because the capacity of the system to provide a good gaming experience had not previously been available. However since I live alone and have a good broadband connection I tried it out and loved it. It’s indistinguishable from playing the game from your own computer.

So, since most of the apps I use are open-source or otherwise available for Linux, I decided to switch from Windows to Ubuntu 24.04. My PC has two hard drives as well as a NVME drive which hosts the Windows system, so I decided to delete and transfer what I could from a 1TB drive and installed Ubuntu on there. It was a breeze and I was quickly up and running. Ubuntu has very good hardware support so most things just worked, and the only thing I had to hunt down was a driver for the Logitech G13 gamepad I constantly use. I’ve been running this for about 3 weeks now and I have no desire to go back.

However, today I came up against an obstacle. On Windows I used Bitvise SSH client to connect to this server, and Bitvise saves its files in its own binary format. I found myself in a situation where I would have to go back to Windows to use Bitvise to connect to my server. Also while my current situation does not require that I have access to a Windows machine, that can change all too easily. So I decided to create a Windows virtual machine (VM from now on), but instead of using a virtual hard drive file, I would simply use the disk on which Windows is already installed. That makes a lot more sense. It’s just more efficient.

I found one set of instructions to help me do that, but it dates back to 2021 and hasn’t been updated more recently, so I spend some time figuring out the more current way to go about it.

How to run a Windows VM from an existing Windows disk

For this you will naturally need a computer running Linux, preferably something Debian-based like Ubuntu. You will also need to install VirtualBox on that computer (downloads here). You should also disable Bitlocker encryption from your Windows drive before proceeding. I didn’t have it enabled on my drive so I don’t know how that would affect the installation.

Set up your user

Your user will need to be part of two groups: “disk” (to enable raw disk access) and “vboxusers”.

sudo usermod -aG disk,vboxusers [user]

Also let’s create a folder “vms” in your home directory in which you will keep your VMs:

mkdir vms

Set up your Windows disk

First, find out where your Windows drive is mounted. By default when you install Ubuntu on a computer with FAT32 or NTFS drives, they will be accessible to Linux.

lshw -short -class disk,volume

This will show you a list of your hard disks and their partitions. Look for a disk that contains a “Windows FAT volume”, a “reserved partition” and one or more “Windows NTFS volumes” and note the entry in the Device column for the disk (not partition). Usually this will be “/dev/nvme0” (if you have a NVME disk) or “/dev/sda”.

Next, we’re going to create a special file that points to that disk using a utility that is installed with VirtualBox. Enter this as your regular user (not root):

VBoxManage createmedium disk --filename=vms/[disk file].vmdk --variant=RawDisk --format=VMDK --property RawDrive=[Windows drive]

This file points at the location of the Windows drive.

Create a VM

Now we’ll start using VirtualBox itself. But before we do, let’s install the VirtualBox Extension Pack (download from here). To install the file just double-click on it on the file. It will launch VirtualBox and the installation will take place.

In the VirtualBox dashboard, click on the Create a new virtual machine (VM) link.

  1. In the VM Name field, enter a name for the virtual machine
  2. The VM Folder should be “/home/[user]/vms”
  3. In the OS field select Microsoft Windows
  4. In the OS Version field select the version of Windows installed on your Windows disk.Win12
  5. Click on the Next button at the bottom right of the New Virtual Machine window.
  6. Under Specify virtual hardware, adjust the Base Memory and Number of CPUs. Bring the Disk Size slider to the lowest value. Note that using 16GB (or more) of RAM is highly recommended otherwise you’ll find the VM experience very taxing, but keep in mind that this memory will not be available to your Linux apps while the VM is running.
  7. Select Use EFI
  8. Click on the Next button.
  9. Click on the Finish button.

Attach The Windows Disk to the VM

Now we’ll attach the pointer file we created in step 1 to the VM.

  1. In the Machines tab of VirtualBox Manager, right-click on the new VM and click on Settings.
  2. In the [VM]-Settings window, select the Storage tab.
  3. You’ll see a .vdi file which we won’t be using. Click on the Add Attachment button at the bottom right of the Devices box (see below) and select Hard disk from the dropdown menu. Attach Disk
  4. In the Hard Disk selector window, click the Add button.
  5. Select the .vmdk file you created earlier in the file selection dialog box.
  6. With the .vmdk file selected, click on the Choose button.
  7. In the [VM]-Settings window, select the .vdi file, and click on the Remove Attachment button (next to the Add Attachment button).
  8. Click OK to save your VM configuration.

Run Your VM

Nothing left to do but to run the VM and make sure it works, so in the Machines tab of VirtualBox Manager, right-click on the VM you just edited and select Start > Start with GUI.

You should be able to log into your Windows installation.

If you have tried in the past to install Linux and modified the UEFI partition of your boot disk… well, you will then have to navigate around the disk using the GRUB CLI to fix your boot sequence. This is beyond the scope of this particular tutorial, but instructions are easily found online. I had to do this myself.

Something else that came to mind as I was writing this was to try and see if I can do the same thing using QEMU instead of VirtualBox, which I will also write a tutorial for if I can manage to do it.

Keep in mind that virtualization at the local level can be a bit tricky and resource-intensive. It’s also one of the rare things that can completely freeze up your system and force you to reboot it — that’s called a kernel panic.

What is a blockchain?

2018 is poised to be year when cryptocurrencies become mainstream. The original cryptocurrency, Bitcoin, has entered the common jargon of the modern world last year as its valuation hit record a record high of nearly 20k USD/BTC, and stayed in the news as its valuation dropped to more reasonable levels. Ethereum is also gaining recognition as it became the #2 cryptocurrency in terms of market capitalization. In short, a little over 8 years since the creation of Bitcoin cryptocurrencies are gaining recognition and acceptance in the “real” world.

Cryptocurrencies are created as part of something called a blockchain. And more than cryptocurrencies, it is the blockchain idea which is expected to have a huge impact on the computing world, at least for the next couple of years. As such it is a good idea to learn what a blockchain is, at both a basic and more advanced level.

The Basics

At its core, a blockchain is a distributed ledger. Those with an accounting background will immediately recognize what a ledger is — it is a record of transactions. A blockchain is distributed, which means that entries in the ledger are written by many parties, as opposed to by one centralized authority.

Like an ordinary paper ledger, blockchains are write-once. Once a block has been verified and added to the blockchain it cannot be erased or modified. This insures that transactions cannot be taken back.

The Nodes

All these “parties” are actually computers running a node for the blockchain’s network on the internet. This involves executing software which contributes to the blockchain network. Depending on the network involved there may be several types of nodes in a blockchain; this will be explored in depth later.

The Blocks

Nodes compile a number of transactions into a block. How large the blocks are, and how often they are verified, varies widely between blockchains. For example, the Bitcoin blockchain generates a block every 10 minutes. The Ethereum blockchain, in comparison, generates a block in less than 20 seconds, and Bitshares blocks are generated every 3 seconds at most. A number of factors affect block time; if you’re not intimidated by math check out this article for more information.

The Chain

Blockchains are so named because each new block is appended to the previous block, effectively forming a chain. In fact one can always look at certain information in the latest block of any given blockchain and trace the blockchain’s history all the way back to its very first block.

Hashing

Since blocks are appended to the blockchain by several different nodes, there needs to be a way to ensure that only the block with the right data can be added at any given time. Otherwise there would be no way of ensuring the continuity of the blockchain from the genesis block to the most current one.

This is where hashing comes in. Hashing is a cryptographical technique that is used to generate a unique code that can be used to identify a set of data, rather like a fingerprint. The hash is generated from the transactions contained in the block and recorded as data in the block, which also includes the hash from the previous block. This is one of the mechanisms used to verify any new blocks. If the previous-block hash does not match the previous block’s recorded hash, then the current block is invalid and cannot be added to the chain.

The actual library used to generate the hashes depends on the blockchain. SHA256 is a popular one and is used by Bitcoin. Other libraries include scrypt, X11, Cryptonight and ETHash.

Hashing produces a completely different string if there is any change whatsoever to the original hashed content. The SHA256 library can produce a very large number of distinct values (3.4028237e+38) so arriving at the same value from two different pieces of content is extremely unlikely. By comparison, the chances of winning the Powerball lottery in the USA is 1 in 2.92e8. One could win this lottery 4 times and that would still be less likely than generating the same hash from 2 different sources. Thus the use of hash values makes blockchains virtually tamper-proof.

This was a very basic overview of blockchains. We’ve barely scratched the surface. In my next few articles I will be providing more in-depth coverage on subjects such as concensus algorithms, blockchain node types, the relationship between blockchains and cryptocurrencies, and how the blockchain can be used by businesses to streamline processes and reduce processing costs.

Lenovo update…

I got a call from Lenovo today informing me that one of the items ordered for me was on back-order… good thing I wasn’t holding my breath. AFAIK it’s the system restore disk for the laptop, which just raises further questions. If I need an additional copy of a CD/DVD I’ve already made before, I just use the image to make a new copy. Does Lenovo not know this?

I’m pretty sure this laptop has been a paperweight for longer than it’s been operating since I bought it.

Lenovo: a fall from grace

I used to be very happy with my Thinkpad X60 Tablet, and was practically an evangelist for the Thinkpad line. Just in the past year I practically sold two systems, a laptop and an iPad-style tablet, for the company. However this good feeling only lasted until my latest purchase. As things stand now I am pretty certain that I will never buy Lenovo again.

My mistake was buying a Thinkpad X220i Tablet in August of 2011. Said laptop stopped working in November, when I tried to start it Windows crashed with a 0x00000e9 error (unspecified I/O error, but more specifically a hard disk problem). Whatever, these things happen. They sent me a set of startup disks (DVDs) to try and make sure the HD was the problem. Those startup disks didn’t work (couldn’t boot off any of them), so I called Lenovo again, they apologized (they’re a very apologetic company, to the point that it becomes annoying really), told me they would send me another set of startup DVDs.

So I waited… and waited, and waited. After a couple of weeks (today) I decided to call and see what was up. The first support call seemed to go well, until I got disconnected. So I called again. The lady took my case number, told me Lenovo had a tracking number for the disks, gave me the tracking number, but it was pretty clear that she was refusing to do anything else (wtf is up with that?). I look up the tracking info through purolator (the most useless delivery company IMHO but that’s another story) and it’s clear that the package was returned to Lenovo. Couldn’t the lazy lady have at least looked that up instead of just wasting my time?

So I call for the third time on that day, and finally I get to talk to someone who knows what he’s doing. He is sending me a new hard disk and recovery disks. Well, hopefully he is. You’ll excuse me for not holding my breath until they get here.

Now, if the problem were only the service, passe encore. However the X220i Tablet is a piece of junk. It feels cheap and flimsy, the weight is unevenly balanced, and the build quality is terrible — I can pull the battery in an out of its housing by a couple of millimeters when it’s in the locked position! What’s more, the touchscreen has never worked properly, and that’s kind of the point of owning a tablet PC. Why does the 5 year old X60T I have feel so much better and more solid? Is it perhaps because it’s an IBM-branded product?

I once heartily recommended Lenovo to friends and family, but I can no longer honestly do so. The quality of the company’s products and service has taken such a nosedive that it’s really not worth endorsing anymore. It’s a shame because Thinkpad used to be the brand to go to for a first-rate experience with laptops, but clearly they’ve taken the road well-travelled of cheaping out on the design, cheaping out on the manufacturing, and cheaping out on the service. All of which I wish I could say to Lenovo customer service, but try as I might I cannot find an actual customer service number for them anywhere. I guess that’s representative of the company’s recent attitude towards the people who give them money…