Powershell Script: Rename Files with Characters that are Invalid for OneDrive for Business and Sharepoint Online

I’m a big fan of Microsoft’s Office 365 offerings, and one of my favorite components of that is the OneDrive storage that comes with each user. It’s a great way to make sure files are always backed up and available anywhere you have an internet connection.

One of the challenges you can encounter when using OneDrive for Business is that not all characters that are valid for filenames in a Windows environment are valid for use in OneDrive (and Sharepoint Online files for that matter). According to this article, the list of invalid characters are: \ / : * ? ” < > | # %.

There are several other restrictions (such as file size and name length), but the invalid characters issue is the one I encounter the most frequently, especially when I migrate a user who has a lot of files to OneDrive for Business. In one case, there were over 1,000 files with invalid characters, and I wasn’t about to rename those by hand.

Looking online, there are several great resources for Powershell scripts to solve this problem, although some were out of date and did not accurately reflect the current restrictions of the files files. In the end, there were two sources I liked a lot and I combined them into a single script and modified that. It’s not a perfect or complete script, and I don’t consider myself even proficient with Powershell. However, this script has worked well for me.

First, the sources:

Use PowerShell to check for illegal characters before uploading multiple files into SharePoint

Fix file names for Skydrive Pro syncing

I liked that the second one outputs to a TSV file that you can open with Excel and review. The first source just prints to the console, and when you have a ton of files, that isn’t very useful. Because of the output to the file, I was able to tweak things a little more, adding in a few additional things, such as searching for files with “%20” in the name (html encoded space).

function Check-IllegalCharacters ($Path, $OutputFile, [switch]$Fix, [switch]$Verbose)
{
    #The maximum allowed number of characters of a file's full path + name
    $maxCharacters = 400
    #The maximum file size
    $maxFileSize = 2147483648
    #A list of file types that can't be sync'd
    $invalidFileTypes = ".tmp", ".ds_store"
    #A list of file names that can't be sync'd
    $invalidFileNames = "desktop.ini", "thumbs.db", "ehthumbs.db"

    Write-Host Checking files in $Path, please wait...

    #Only run for a valid path
    if (!(test-path $path))
    {
        'Invalid path for file renames'
    }
    else
    {
        #if the output file exists empty it first
        if (test-path $outputFile)
        {
            clear-content $outputFile
        }
        #add headers to output file
        Add-Content $outputFile "File/Folder Name	New Name	Comments";

        #Get all files and folders under the path specified
        $items = Get-ChildItem -Path $Path -Recurse
        foreach ($item in $items)
        {
            #Keep a flag to indicate whether or not we can perform the updates (some problems are deal breakers)
            $valid = $true
            #Keep an array list for comments
            $comments = New-Object System.Collections.ArrayList
            
            if ($item.PSIsContainer) { $type = "Folder" }
            else { $type = "File" }
        
            #Check if item name is longer than the max characters in length
            if ($item.Name.Length -gt $maxCharacters)
            {
                [void]$comments.Add("$($type) $($item.Name) is $($item.Name.Length) characters (max is $($maxCharacters)) and will need to be truncated")
                $valid = $false
            }

            if($item.Length -gt $maxFileSize)
            {
                [void]$comments.Add("$($type) $($item.Name) is $($item.Length / 1MB) MB (max is $($maxFileSize / 1MB)) and cannot be synchronized.")   
                $valid = $false
            }

            if($invalidFileNames.Contains($item.Name))
            {
                [void]$comments.Add("$($type) $($item.Name) is not a valid filename for file sync.")
                $valid = $false
            }

            if($invalidFileTypes.Contains($item.Name.Substring($item.Name.Length-4)))
            {
                [void]$comments.Add("$($type) $($item.Name) type $($item.Name.Substring($item.Name.Length-4)) is not a valid file type for file sync.")
                $valid = $false
            }
           
            #Technically all of the following are illegal \ / : * ? " < > | # %
            #However, all but the last two are already invalid Windows Filename characters, so we don't have to worry about them
            $illegalChars = '[#%]'
            filter Matches($illegalChars)
            {
                $item.Name | Select-String -AllMatches $illegalChars |
                Select-Object -ExpandProperty Matches
                Select-Object -ExpandProperty Values
            }
            
            #Replace illegal characters with legal characters where found
            $newFileName = $item.Name
            Matches $illegalChars | ForEach-Object {
                if($Verbose){ [void]$comments.Add("Illegal string '$($_.Value)' found") }
                #These characters may be used on the file system but not SharePoint
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "-") }
                if ($_.Value -match "%20") { $newFileName = ($newFileName -replace "%20", " ") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "-") }
            }

            if($comments.Count -gt 0)
            {
                #output the details
                Add-Content $outputFile "$($item.FullName)	$($item.FullName -replace $([regex]::escape($item.Name)), $($newFileName))	$($comments -join ', ')"    
                if($Verbose)
                { 
                    Write-Host $($type) $($item.FullName): $($comments -join ', ') -ForegroundColor Red
                }
            }
                
            #Fix file and folder names if found and the Fix switch is specified
            if ($newFileName -ne $item.Name)
            {
                if($fix -and $valid)
                {
                    Rename-Item $item.FullName -NewName ($newFileName)
                    if($Verbose)
                    {
                        Write-Host $($type) $($item.Name) has been changed to $($newFileName) -ForegroundColor Yellow
                    }
                }
            }
        }
    }
    Write-Host "Done"
}

#Example: Check-IllegalCharacters -Path 'C:\Users\User\Downloads\Files With Errors' -OutputFile 'C:\Users\User\Desktop\RenamedFiles.tsv' -Verbose -Fix

I noticed that I ran into some errors when files were deeply nested – for example if you have to rename a file in a folder that also was renamed. Re-running the script a few times fixed that problem for me (each time it would fix one more layer of folders – so if you had a file inside 3 levels of folders that also needed renaming, it would take four passes in order to complete everything.

Quick Tip: How to get a text list of files in a Windows Directory

Every once in a while I need to get a list of just the file names in a directory. Sometimes there are a LOT of files and it would be a pain in the ass to type them out or write a program to manipulate them in some way.

The quick way to do this in Windows (images below are for Windows 10) is as follows:

  1. Open a windows explorer window (shortcut: hit Win + e, where “Win” is the Windows key) and navigate to the folder containing the files you want to print out
  2. Hold the Shift key and right-click in the folder (don’t click on any actual files) and choose the option “Open Powershell Window here”

  3. In the powershell window that opens, type

    Get-ChildItem -name

That’s it! now it prints a list of file names that you can copy/paste from the powershell window into something else (Excel, notepad, word, whatever)

Alternate method: command prompt

  1. Open a command prompt (Hit the Windows key, type “cmd” and hit enter)
  2. Navigate to the folder containing the images (type cd “<your file path>”, for example: “C:\Users\Public\Public Pictures\Sample Pictures”)
  3. Type the command below:

    dir /b

  4. If you’re using Windows 10, you can simply copy/paste from the command prompt. With an older version of windows, you need to:
    1. Right click in the command window and choose “Select All”
    2. Hit the Enter key. This will copy the contents of the command prompt to your paste buffer so you can paste using ctrl + v or right-click -> Paste