Powershell Script: Rename Files with Characters that are Invalid for OneDrive for Business and Sharepoint Online

I’m a big fan of Microsoft’s Office 365 offerings, and one of my favorite components of that is the OneDrive storage that comes with each user. It’s a great way to make sure files are always backed up and available anywhere you have an internet connection.

One of the challenges you can encounter when using OneDrive for Business is that not all characters that are valid for filenames in a Windows environment are valid for use in OneDrive (and Sharepoint Online files for that matter). According to this article, the list of invalid characters are: \ / : * ? ” < > | # %.

There are several other restrictions (such as file size and name length), but the invalid characters issue is the one I encounter the most frequently, especially when I migrate a user who has a lot of files to OneDrive for Business. In one case, there were over 1,000 files with invalid characters, and I wasn’t about to rename those by hand.

Looking online, there are several great resources for Powershell scripts to solve this problem, although some were out of date and did not accurately reflect the current restrictions of the files files. In the end, there were two sources I liked a lot and I combined them into a single script and modified that. It’s not a perfect or complete script, and I don’t consider myself even proficient with Powershell. However, this script has worked well for me.

First, the sources:

Use PowerShell to check for illegal characters before uploading multiple files into SharePoint

Fix file names for Skydrive Pro syncing

I liked that the second one outputs to a TSV file that you can open with Excel and review. The first source just prints to the console, and when you have a ton of files, that isn’t very useful. Because of the output to the file, I was able to tweak things a little more, adding in a few additional things, such as searching for files with “%20” in the name (html encoded space).

function Check-IllegalCharacters ($Path, $OutputFile, [switch]$Fix, [switch]$Verbose)
{
    #The maximum allowed number of characters of a file's full path + name
    $maxCharacters = 400
    #The maximum file size
    $maxFileSize = 2147483648
    #A list of file types that can't be sync'd
    $invalidFileTypes = ".tmp", ".ds_store"
    #A list of file names that can't be sync'd
    $invalidFileNames = "desktop.ini", "thumbs.db", "ehthumbs.db"

    Write-Host Checking files in $Path, please wait...

    #Only run for a valid path
    if (!(test-path $path))
    {
        'Invalid path for file renames'
    }
    else
    {
        #if the output file exists empty it first
        if (test-path $outputFile)
        {
            clear-content $outputFile
        }
        #add headers to output file
        Add-Content $outputFile "File/Folder Name	New Name	Comments";

        #Get all files and folders under the path specified
        $items = Get-ChildItem -Path $Path -Recurse
        foreach ($item in $items)
        {
            #Keep a flag to indicate whether or not we can perform the updates (some problems are deal breakers)
            $valid = $true
            #Keep an array list for comments
            $comments = New-Object System.Collections.ArrayList
            
            if ($item.PSIsContainer) { $type = "Folder" }
            else { $type = "File" }
        
            #Check if item name is longer than the max characters in length
            if ($item.Name.Length -gt $maxCharacters)
            {
                [void]$comments.Add("$($type) $($item.Name) is $($item.Name.Length) characters (max is $($maxCharacters)) and will need to be truncated")
                $valid = $false
            }

            if($item.Length -gt $maxFileSize)
            {
                [void]$comments.Add("$($type) $($item.Name) is $($item.Length / 1MB) MB (max is $($maxFileSize / 1MB)) and cannot be synchronized.")   
                $valid = $false
            }

            if($invalidFileNames.Contains($item.Name))
            {
                [void]$comments.Add("$($type) $($item.Name) is not a valid filename for file sync.")
                $valid = $false
            }

            if($invalidFileTypes.Contains($item.Name.Substring($item.Name.Length-4)))
            {
                [void]$comments.Add("$($type) $($item.Name) type $($item.Name.Substring($item.Name.Length-4)) is not a valid file type for file sync.")
                $valid = $false
            }
           
            #Technically all of the following are illegal \ / : * ? " < > | # %
            #However, all but the last two are already invalid Windows Filename characters, so we don't have to worry about them
            $illegalChars = '[#%]'
            filter Matches($illegalChars)
            {
                $item.Name | Select-String -AllMatches $illegalChars |
                Select-Object -ExpandProperty Matches
                Select-Object -ExpandProperty Values
            }
            
            #Replace illegal characters with legal characters where found
            $newFileName = $item.Name
            Matches $illegalChars | ForEach-Object {
                if($Verbose){ [void]$comments.Add("Illegal string '$($_.Value)' found") }
                #These characters may be used on the file system but not SharePoint
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "-") }
                if ($_.Value -match "%20") { $newFileName = ($newFileName -replace "%20", " ") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "-") }
            }

            if($comments.Count -gt 0)
            {
                #output the details
                Add-Content $outputFile "$($item.FullName)	$($item.FullName -replace $([regex]::escape($item.Name)), $($newFileName))	$($comments -join ', ')"    
                if($Verbose)
                { 
                    Write-Host $($type) $($item.FullName): $($comments -join ', ') -ForegroundColor Red
                }
            }
                
            #Fix file and folder names if found and the Fix switch is specified
            if ($newFileName -ne $item.Name)
            {
                if($fix -and $valid)
                {
                    Rename-Item $item.FullName -NewName ($newFileName)
                    if($Verbose)
                    {
                        Write-Host $($type) $($item.Name) has been changed to $($newFileName) -ForegroundColor Yellow
                    }
                }
            }
        }
    }
    Write-Host "Done"
}

#Example: Check-IllegalCharacters -Path 'C:\Users\User\Downloads\Files With Errors' -OutputFile 'C:\Users\User\Desktop\RenamedFiles.tsv' -Verbose -Fix

I noticed that I ran into some errors when files were deeply nested – for example if you have to rename a file in a folder that also was renamed. Re-running the script a few times fixed that problem for me (each time it would fix one more layer of folders – so if you had a file inside 3 levels of folders that also needed renaming, it would take four passes in order to complete everything.

Quick Tip: How to get a text list of files in a Windows Directory

Every once in a while I need to get a list of just the file names in a directory. Sometimes there are a LOT of files and it would be a pain in the ass to type them out or write a program to manipulate them in some way.

The quick way to do this in Windows (images below are for Windows 10) is as follows:

  1. Open a windows explorer window (shortcut: hit Win + e, where “Win” is the Windows key) and navigate to the folder containing the files you want to print out
  2. Hold the Shift key and right-click in the folder (don’t click on any actual files) and choose the option “Open Powershell Window here”

  3. In the powershell window that opens, type

    Get-ChildItem -name

That’s it! now it prints a list of file names that you can copy/paste from the powershell window into something else (Excel, notepad, word, whatever)

Alternate method: command prompt

  1. Open a command prompt (Hit the Windows key, type “cmd” and hit enter)
  2. Navigate to the folder containing the images (type cd “<your file path>”, for example: “C:\Users\Public\Public Pictures\Sample Pictures”)
  3. Type the command below:

    dir /b

  4. If you’re using Windows 10, you can simply copy/paste from the command prompt. With an older version of windows, you need to:
    1. Right click in the command window and choose “Select All”
    2. Hit the Enter key. This will copy the contents of the command prompt to your paste buffer so you can paste using ctrl + v or right-click -> Paste

 

Connecting to Synology DiskStation from Windows 10

I’ve owned a small Synology Diskstation for a few years and really love its features and capabilities, especially considering its cost. One of the primary roles of my DiskStation is to backup my home computers. After recently purchasing a Surface Pro 4 and applying the Creators’ update, I was having trouble connecting to my DiskStation running DiskStation 6.1. After reading quite a few posts online about different problems, it seems the solution I needed was really quite basic.

What was most curious about this problem was that I could not see my DiskStation appear under “Network” under Windows explorer, and I received error code 53 (system error 53 has occurred. The network path was not found) when I tried to map the network drive using the command prompt like so:

net use T: \\DiskStation

Performing nbtstat -c from the command line and net view both listed my DiskStation with the UNC I was expecting, and the correct IP (I have mine configured as a static) in the case of the nbtstat command.

First, I made sure the SMB settings on the DiskStation were set to allow from SMB 1.0 to SMB 3.0 (DiskStation Control Panel -> File Services -> SMB -> Advanced Settings -> Maximum/Minimum SMB Protocol Settings).

Then, in Windows, if I opened Explorer and navigated to the IP address or the UNC sharename (\\DiskStation, for example), it would prompt me for a password. This was the primary point of failure for me earlier – I had forgotten that when logging into another server, whether it’s a Synology DiskStation or a Windows Server, you have to provide the server name AND the account name (or domain name/Account name, in the even that you’re connected to a domain).

So the Username wasn’t just “MyUsername” it was “DiskStation\MyUserName”. Once I did that, my DiskStation appeared under Network.

Killing all connections to a SQL Server Database

One issue I’ve run across frequently during development is restoring a database to a newer state. Often, when I want perform the restore, there are active connections to my development database, so restoring will fail.

Of course, StackOverflow had the answer to this, but I’ve searched for the solution to this problem enough times where it made sense for me to finally write it down.

Script to kill all connections to a database (More than RESTRICTED_USER ROLLBACK)

User AlexK posted this excellent solution:

For MS SQL Server 2012 and above

USE [master];

DECLARE @kill varchar(8000) = '';  
SELECT @kill = @kill + 'kill ' + CONVERT(varchar(5), session_id) + ';'  
FROM sys.dm_exec_sessions
WHERE database_id  = db_id('MyDB')

EXEC(@kill);

For MS SQL Server 2000, 2005, 2008

USE master;

DECLARE @kill varchar(8000); SET @kill = '';  
SELECT @kill = @kill + 'kill ' + CONVERT(varchar(5), spid) + ';'  
FROM master..sysprocesses  
WHERE dbid = db_id('MyDB')

EXEC(@kill); 

 

Fix: Visual Studio doesn’t remember last open documents

After installing Visual Studio 2017 a few months back, I noticed that some projects were loading strangely, while others loaded just fine. The two main issues I experienced were:

  • Documents I had open on my previous run of VS wouldn’t load upon running the Visual Studio 2017 application
  • Windows I had arranged in my multi-monitor layout were not loading where I expected them

A quick Stack Overflow search led me to the answer regarding the first: the .suo file had become corrupt. Once I knew that, the trick was finding the .suo file:

  1. From the directory containing your solution file (.sln), open the folder named “.vs”.
  2. In the “.vs” folder, open the folder that has a name matching your solution name.
  3. Inside the solution folder, there may be multiple folders, one for each version of Visual Studio
    1. v14 is for Visual Studio 2015
    2. v15 is for Visual Studio 2017

These folders will contain your .suo file, which is hidden by default in Windows, so you need to enable “Show hidden files, folders, and drives” in your Folder options in order to see it. For instructions on that (Win 7,8, or 10), see the following article: https://www.howtogeek.com/howto/windows-vista/show-hidden-files-and-folders-in-windows-vista/

I still haven’t found a solution to my second issue (I will definitely write about it if I find one).

What the SUO (Solution User Options) file controls

After solving my problem, I decided to take a look at the responsibilities of the .suo file. Microsoft’s documentation (VS 2015 version – 2017 isn’t available at the time of this writing) isn’t very forthcoming in detailing what exactly the SUO is doing. Based on digging around on the web, it seems that the following are its responsibilities (among others):

  • Remembers last open files
  • Remembers breakpoints
  • Remembers expanded nodes in solution explorer
  • Remembers startup project
  • Remembers last open tool windows and their positions
  • Remembers watch window contents

The file is encoded and not human-readable, so it’s not something you can simply hack around with like you can a solution (.sln) or project (.xxproj) file. It should not be added to version control.

How to Fix credential validation issue on Azure WebJob renewal of Let’s Encrypt Certificate

A while back, I posted about setting up SSL encryption for free with Azure and Let’s Encrypt: Let’s Encrypt + Azure = Win!

This has been working smoothly for me since I set it up, but I noticed that errors started popping up in the log recently. Here is part of the stack trace:

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.RenewCertificate —> Microsoft.IdentityModel.Clients.ActiveDirectory.AdalServiceException: AADSTS70002: Error validating credentials. AADSTS50012: Invalid client secret is provided. Trace ID: 958b11ab-839d-4a8d-97e6-fad1c3df0300 Correlation ID: e3f7c035-8978-4aa2-b01a-5c8fc74661ac Timestamp: 2017-05-31 14:14:26Z —> System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.GetResponse() at Microsoft.IdentityModel.Clients.ActiveDirectory.HttpWebRequestWrapper.

It turns out that the API Key I had setup for my application registration had expired. I had to create a new key with no expiration and then update my Web Applications’ settings with the new Client secret. The exact steps I took are listed below:

  1. Login to Azure
  2. Navigate to “App Registrations”
  3. Choose the Registration you need to update
  4. Click the “settings” icon (or “All Settings” button)
  5. Choose “Keys” under API Access
  6. Type a description into the new row, choose “Never” under the duration drop down and then hit “Save” above.
  7. Once saved, copy the value (it won’t be visible again if you don’t copy it now)
  8. (Optional) delete your old key
  9. Navigate to the Azure App Service that has the web job that registers your SSL certificate
  10. Choose “Application Settings” from the menu
  11. Scrolling down to where you have a setting titled something like “letsencrypt:ClientSecret” (assuming you did the setup as in the article linked at the top) and paste the value you copied into the second text box
  12. Click “Save” above

Once you’re done, the web job should work the next time it runs. For another explanation with some pictures of the process, check out this blog post here: Let’s Encrypt on Azure Web Apps – Key Expiration Issue.

Converting a Lead Acid Battery-Powered Lawn Mower to Use Lithium Batteries

About 7 years ago, I was in the market for a new lawn mower. Looking at all the options at the time, I decided to go with an electric 24v, 20 amp-hour lawn mower sold under a brand called Earthwise. Here is the lawn mower in all its glory, model 60120:

Earthwise 60120 electric lawn mower

Credit: Amazon.com

I loved this mower from the get-go. It was extremely quiet, could mow my entire 1/4″ acre lawn in a single charge, and didn’t require gas, oil, spark plugs, etc. The only maintenance to do was charge the battery and sharpen the blade.

That entire first season was great, but it wouldn’t hold a charge for nearly as long the second season. I started having to charge it two times to finish my lawn by the fall. The third season was even worse. It wouldn’t hold a charge for more than a few minutes.

Opening the battery compartment on the lawn mower. 2, 12 volt batteries are wired in series to produce the 24 volts that power the mower.

I knew the batteries needed to be replaced, but I had no idea how much they would cost. I think I paid something like $150 for a replacement set, which is a pretty steep price. A few years later, those batteries were dead too. I gave up on it and bought a cheap used gas mower last year, but I hated using it. The pull starter was finicky, it would occasionally expel clouds of black smoke, and I would forget to buy gas for it from time to time.

I decided to look around and see if other people had found solutions, and sure enough they had. With the proliferation of lithium-ion batteries, it isn’t hard to find the batteries needed or to perform the upgrade.

What I needed to Do

In a nutshell, the task was simple. I had to do the following:

  • Buy lithium batteries to replace the lead-acid batteries
  • Cut the ends off of the black and white wires coming from the top of the battery case.
  • Solder new connectors to the black and white wires (whatever connectors matched the batteries I would buy)

It’s really that simple – just a few tasks and I would be on my way. A little research was required to figure out what batteries to buy, however.

Lithium Polymer (aka Li-Po, LiPo, or Li-Poly) Batteries

Lithium Polymer is a bit of a misnomer, since Lithium Polymer batteries are technically just lithium-ion batteries in a polymer casing (check out this excellent article for a good explanation on the difference between lithium-ion and lithium-polymer: Lithium Polymer vs Lithium-Ion batteries: What’s the deal?), but they came highly recommended as the battery of choice for this project. These batteries are being used all over the hobby world today, with drones leading the way. Lithium Polymer batteries are also used in many computers and cellphones.

Lithium Polymer batteries have a few important pieces of information written on them:

  • Voltage: You need a voltage that closely matches the mower. Since my lawn mower’s voltage is 24 volts, a 22.2v li-po battery is the best fit. Lithium polymer cells have a “nominal” voltage of 3.7v. Lithium polymer battery voltages are just multiples of 3.7v because they run multiple cells together to form a single battery. Therefore a 22.2v battery is really made up of 6 3.7v cells. Nominal voltage means the mid-range voltage because the cells run at 4.2v when fully charged and 3.2v when fully discharged. That means a 22.2v battery will output somewhere between 19.2v and 25.2v during the course of its run
  • Number of cells: Batteries will often have something like “6S” or “3S” printed on them. This corresponds to the number of cells in the battery. 6S = 6 cells = 22.2v. 3S = 3 cells = 11.1v.
  • Capacity/Runtime/Amp hours: Runtime is measured in mAh aka milliamp hours. A battery that has 5000 mAh has a runtime of 5 amp hours. Considering my mower had 20 amp hours, I want my batteries to try and match that if I want the same amount of runtime.
  • “C” Rating/Capacity Rating/Discharge rating: Batteries also list a C rating, which is used to determine the maximum load that a battery can safely sustain. 1C = the capacity of the battery. Therefore if a battery has 5 amp hours/5000mAh, 1C = 5amps. If a battery’s C rating is 40C, then the max is 200amps.

All of this is explained in much greater detail by this excellent article: A Guide to Understanding LiPo Batteries

Based on all this information, I knew I needed 22.2v batteries, and I wanted to get somewhere around 20 amp hours. I read from another resource that 20C was sufficient for others who did this project, so I figured I could do that or above. Looking online, I found the batteries to be fairly expensive. I settled on 2 pairs of these batteries (sold as 2 each): https://www.amazon.com/gp/product/B01AW7CKLW/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1 (22.2v, 4500mAh, 6S, 45C, Deans connector). Note that it says they come with XT-60 connectors, but the picture shows Deans connectors, which is what I received.

Deans Connectors

Deans connectors are apparently very common in the hobby world. I bought a pack of male plugs and a few splitters:

Soldering the ends was a little bit tricky as the connectors from the battery were fairly thick. I eventually got it right though, and the connections work fine.

Other Considerations

Charging the Batteries

You also need a charger for these batteries. Unfortunately, you have to charge them one at a time, so if you want 4 batteries like I have, you either want a multi-battery charger or you have to be a little patient.

Knowing when to Charge

It’s a good idea to get low-voltage indicators: https://www.amazon.com/gp/product/B003Y6E6IE/ref=od_aui_detailpages01?ie=UTF8&psc=1. If you put these on your batteries when you use them, they make a rather annoying sound when the voltage drops to the low threshold. This is important because your mower’s meter isn’t going to tell you when your charge is low. If you push a li-po battery too much, you can cause damage to the battery or it could explode. These things are loud enough that I can hear them while running the lawn mower.

Safely Storing and Transporting

Li-Pos are very flammable and difficult to put out. It is advised to buy a (relatively cheap) fireproof bag for storage and charging and that you charge using the “Storage” setting when you aren’t going to use them for a week or more. You should store them at room temperatures and it is advised you are present while charging due to the fire hazards. The fireproof bag I purchased is here: https://www.amazon.com/gp/product/B01H4QCZ4G/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

Conclusion

The mower now holds a charge that is easily long enough to mow my entire lawn again. Li-Po batteries are supposed to last for 200-300 charges in good conditions, so I’m hoping to get several years out of this setup. Charging is a bit of a pain, but I tend to mow on the weekends, so I’m usually around long enough to charge all 4 of them (It takes a couple of hours to fully charge each battery).

The mower has enough power to mow at most of the height settings, but it struggles to mow at the lowest levels. This is fairly consistent with how the lead acid batteries performed as well – there just isn’t enough output to chew up thick grass that is significantly lower than the current height.

It turned out that doing this conversion was fairly easy, but not particularly cheap. All told, I bought the following:

  • 2 sets of 2 x 22.2v 4500mAh, 45C batteries – $110 each set ($220 total)
  • 2 battery low-voltage indicators – $5 each ($10 total)
  • Li-Po battery charger/balancer – $55
  • Fireproof bag (holds 4 batteries, came with 2 more low-voltage indicators) – $15

Together, that’s $300, which could buy a decent gas mower. However, I’m a nerd so I enjoyed the project.

Resources:

Configuring a Fraud Detection Whitelist on Office 365 / Exchange 365

I’ve been setup with Office 365 for around a year, and I’m still discovering little things to tweak and optimize. One such thing I ran across today was a little message in some emails that were generated by an on-premises web server:

This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing

While the link provided by Microsoft about spoofing describes spoofing in detail, it doesn’t say anything about when you know something isn’t fraudulent and want to prevent it from flagging Exchange. After doing a little digging (I started by looking for some kind of a whitelist or whitelisting certain IP addresses on Exchange/Office 365), I came across a very helpful article: This Sender Failed Our Fraud Detection Checks and May Not Be Who They Appear to Be.

In a nutshell, the problem is that the email headers specify an origination IP address (our web server’s address) that isn’t allowed as part of Exchange’s SPF (Sender Policy Framework) configuration. The SPF configuration will examine an email’s domain and ensure that the domain matches an allowed list, to prevent fraudulent sending. After all, it’s pretty damn easy to spoof an email address using software.

SPF filters are added to your DNS records, and are pretty easy to update. To that end, I logged into my DNS provider and took a look at the records for my domain. There, I found a TXT record that was setup when I initially configured Office 365. This record had the following value:

v=spf1 include:spf.protection.outlook.com -all

In order to add my web server’s address to this record and thus resolve my issue, the line simply needed to be modified as such:

v=spf1 ip4:xxx.xxx.xxx.xxx include:spf.protection.outlook.com -all

xxx.xxx.xxx.xxx is, of course, the IP address you want to “whitelist.”

Once the old DNS record expires (I have a TTL of 1 hour on this record), the new configuration should take effect and your messages will no longer be destined for your Junk Email folder.

EntityFramework – Grouping by Date Ranges

If you’ve ever created an outstanding balance report or other report that deals with aggregating data into date ranges, you’ll know that it isn’t immediately obvious how to structure your query, whether using SQL or LINQ (at least, it wasn’t to me).

My initial thought was to run multiple queries (one for each time range) and munge the results together. However, an elegant solution is to use SQL’s CASE expression to group date ranges together.

Let’s say you wanted a report that summed the amount of unpaid invoices in 20 day groupings (0-19 days past due, 20-39 past due, 40+). You could write something like this:

SELECT DaysSinceDueRange, SUM(Amount)
FROM (SELECT CASE WHEN GETDATE() - DueDate < 20 THEN 0
		  WHEN GETDATE() - DueDate BETWEEN 20 AND 39 THEN 20
		  WHEN GETDATE() - DueDate > 39 THEN 40
             END AS DaysSinceDueRange,
             Amount
       FROM Invoices
       WHERE Unpaid=1) inv
GROUP BY DaysSinceDueRange

This is really elegant, but then the question becomes how to do this with an ORM like EntityFramework. There are a couple of tricks required here:

  1. To do the date comparisons, EntityFramework requires the usage of System.Data.Entity.DbFunctions.DiffDays method (in EF 6 – it used to be in System.Data.Objects.EntityFunctions). If you try to do something like (DateTime.Now – invoice.DueDate).TotalDays, you’ll get an exception “DbArithmeticExpression arguments must have a numeric common type” because the subtraction operator is not defined for Dates in SqlServer.
  2. To do CASE / WHEN / THEN / END in EntityFramework, you have to make use of a lot of ternary operators. It can be kind of ugly, but if you write your code well enough, it should be fairly readable (or at least as readable as the SQL expression).

Here is an example of the SQL above translated into LINQ:

Context.Set<Invoice>()
       .Where(inv => inv.Unpaid)
       .Select(inv => new
       {
           DaysSinceDueRange = DbFunctions.DiffDays(inv.DueDate, DateTime.Today) < 20 ? 0 :
                               DbFunctions.DiffDays(inv.DueDate, DateTime.Today) >= 20 && 
                                   DbFunctions.DiffDays(inv.DueDate, DateTime.Today) < 40 ? 20 : 
                               40,
           Amount = inv.Amount
       }).GroupBy(inv => inv.DaysSinceDueRange)
       .Select(g => new
       {
           DaysSinceDueRange = g.Key.DaysSinceDueRange,
           Amount = g.Sum(inv => inv.Amount)
       });

Of course, you can get more complicated in a hurry, but I think this is a pretty elegant way to handle grouping data by date ranges.

Project Fi is the way to go if you are a low cellular data user

After years of being an AT&T mobile customer, dating back to the Cingular days, I finally made the jump to Google’s Project Fi last December. All in all, the service has been very good, and the savings have been ridiculous. AT&T and Verizon have recently rolled out unlimited data plans, so the pricing is a little different than the plan I was on, but it’s not too dissimilar from what I had. Note that I do not recommend project Fi if you’re a really heavy cellular data user (> 6GB total) because they charge $10/GB. If you’re using a lot of data, that will add up fast. However, the problem with most plans is they don’t give you anything if you don’t use that data (maybe you get rollover data, but I’d rather have money). Project Fi pays you back for what you don’t use.

With AT&T, My wife, my sister, and I had a family share plan with 6GB of data. The total for this plan was right around $210/mo, and that was with a corporate discount applied. All of us had new-ish smartphones (my wife and sister had iPhones, myself an android), and were coming up on the end of our 2-year contracts.

During my attendance of That Conference last year, I heard someone tell me about Project Fi and how little it cost. I started looking into it and when my phone (LG G3) turned itself into an unbootable brick one day in September, I decided to buy a Nexus 6P as a replacement. Not only was the phone reasonably priced, but was one of the very limited selection of phones that work on Project Fi. It took a little convincing of my wife to move from her iPhone to Android, but when they announced the Pixel, she agreed to make the move.

Cell Coverage and Quality

A few months in, I can tell you the service, at least where I live in Madison, Wisconsin, has been very good. I haven’t had many instances where I couldn’t get a signal. From the Project Fi FAQ:

Project Fi has partnered with Sprint, T-Mobile, and U.S. Cellular, three of the leading carriers in the US, to provide our service.

They also provide a link to a coverage map: https://fi.google.com/coverage

Project Fi also tries to utilize WI-FI calling when there is low cell quality. I have found this to be a bit of a mixed bag – I sometimes don’t get a dialtone or the phone doesn’t indicate that a call is going out until, suddenly, someone picks up.

The only places where I’ve noticed signal quality problems so far have been inside airports. In particular, it was difficult to get a signal at O’hare. Other people I was travelling with who had Verizon received a better signal.

I have also noticed that sometimes SMS messages won’t come through unless I enable cellular data (and yes, I have verified this even when messages are not MMS). My wife hasn’t had the same problem on her pixel, so mine could be a hardware issue or something specifically related to the Nexus 6P.

Costs

Getting back to the costs, our bill comes in at around $45/month. For two people. Previously, it was costing about $140/month for two people. That’s almost $100/mo, we’re saving. Because Fi reimburses you for unused data, it incentivizes us to use less than the 2GB we pay for. I even have a little widget on my phone that shows how much data I use, and it really encourages me to think about how I’m using data.

Here is a breakdown of the charges from my last bill:

Last month’s usage (for Feb 2 – Mar 2)
Unused data Credit for 1.666 GB at $10/GB -$16.65
Next month’s charges (for Mar 2 – Apr 2)
Fi Basics 2 people, $20 + $15/member $35.00
Prepaid data 2 GB at $10/GB $20.00
Taxes & regulatory fees $6.12

Total: $44.47

International Calling

One of the other great benefits of Project Fi is the international calling aspect of the plan. Data is still $10/GB in 135 countries. From their FAQ:

Project Fi offers high speed data in over 135+ countries and destinations for the same $10/GB you pay in the U.S. For a complete breakdown of specific countries please check our International Rates.

Further:

Unlimited international texts are included in your plan. If you’re using cell coverage, calls cost 20¢ per minute. If you’re calling over Wi-Fi, per-minute costs vary based on which country you’re calling and you’re charged only for outbound calls. Please check our international rates for more information.

I haven’t had a chance to try it out, but I love this part of the plan. When I traveled to Belize last year, we paid $20 for a SIM card and calls there are generally very expensive. Data is incredibly expensive. Had I had Project Fi at the time, those charges would have been very minimal (coverage is another issue altogether, but at least when you have coverage, usage comes at a reasonable rate).

Summary

So, in summary, if you’re a relatively low cellular data user and don’t mind having Google phones, this plan is a great value. I’m looking at saving nearly $1200 this year because of it. I can think of a lot better things to do with my money than spend it on cellular service.