You may ask yourself, what is the purpose of this article? Why convert a Windows file to adapt to a UNIX environment like Linux? Isn’t Linux all-powerful? The exceptional capabilities of the Linux operating system do not spare it from incompatible displays of files transferred from other computing platforms.
Just because you can open a file on a Linux environment does not imply that you have full control over how the file’s texts should be displayed.
[ You might also like: How to Find Files Containing Specific Text String in Linux ]
You will encounter instances where a file’s texts or words are jammed together on a single giant line. In other instances, the same file texts’ displays might not have line breaks or sentence spacing.
A common attribute of raw Windows files opened in UNIX systems like Linux is the unavoidable end-of-line display of ^M or Ctrl-M characters.
This article guide seeks to achieve one objective; the conversion of a Windows File to a UNIX file without changing the format of the resulting file.
Ways to Convert Windows File to Unix Format in Linux
We can achieve the objective of our article through several methods. These methods allow us to convert a Windows file to a UNIX file and still retain the original format of the Windows file.
Convert Windows File to Unix Using dos2unix Command
Depending on your Linux operating system distribution, you can install the dos2unix command-line tool from one of the following commands:
$ sudo apt-get install dos2unix [On Debian, Ubuntu and Mint] $ sudo yum install dos2unix [On RHEL/CentOS/Fedora and Rocky Linux/AlmaLinux] $ sudo emerge -a sys-apps/dos2unix [On Gentoo Linux] $ sudo pacman -S dos2unix [On Arch Linux] $ sudo zypper install dos2unix [On OpenSUSE]
The command syntax for using the dos2unix tool is as follows:
$ dos2unix Your_Windows_File Final_Unix_File
So if you have a sample file created on a Windows computing system and want to open it on a Linux computing system without compromising its format, you would use the following command.
$ dos2unix windows_readme.txt unix_readme.txt
Before we run the above command, we need to create a blank unix_readme.txt file that will accommodate the converted file.
$ touch unix_readme.txt $ dos2unix windows_readme.txt unix_readme.txt
As per the screen capture, your converted Windows file should comfortably adapt to any Unix environment.
Using AWK Command – Convert Windows File to Unix
The awk command is pre-installed on all modern full-fledged UNIX computing systems like Linux. To convert our sample Windows file, we would implement the awk command in the following manner:
$ awk '{ sub("r$", ""); print }' windows_readme.txt > new_unix_readme.txt
As you have noted, with the awk command, we don’t need a pre-existing blank Linux file to accommodate the converted Windows file. The command creates and populates the Unix file version of the Windows file on its own.
Using tr Command – Convert Windows File to Unix
If most of the Windows files you open on your Linux environment have unnecessary Ctrl-Z and carriage return characters, then you will appreciate what the tr command has to offer.
Supposing our sample Windows file is a victim of such characters, removing them will require implementing the following command:
$ tr -d '' < windows_readme.txt > polished_unix_readme.txt
The inbuilt nature of the tr command also generates the resulting UNIX file without the need for its pre-creation.
The flexibility of the three discussed approaches to converting any editable Windows file to UNIX file format should save you from the headaches of having to manually edit your downloaded or transferred Windows files to remove unwanted characters and spaces while on a Linux environment.
Overview
The format of Windows and Unix text files differs slightly. In Windows, lines end with both the line feed and carriage return ASCII characters, but Unix uses only a line feed. As a consequence, some Windows applications will not show the line breaks in Unix-format files. Likewise, Unix programs may display the carriage returns in Windows text files with Ctrl-m (^M) characters at the end of each line.
There are many ways to solve this problem. This document provides instructions for using FTP, screen capture, unix2dos and dos2unix, tr, awk, Perl, and vi to do the conversion. To use these utilities, the files you are converting must be on a Unix computer.
Note:
In the instructions below, replace unixfile.txt with the name of your Unix file, and replace winfile.txt with the Windows filename.
FTP
When using an FTP program to move a text file between Unix and Windows, be sure the file is transferred in ASCII format, so the document is transformed into a text format appropriate for the host. Some FTP programs, especially graphical applications, do this automatically. If you are using command line FTP, before you begin the transfer, enter:
ascii
Note:
For file transfers to and from UITS systems at Indiana University, you must use an SFTP client.
tr
You can use tr to remove all carriage returns and Ctrl-z (^Z) characters from a Windows file:
tr -d '1532' < winfile.txt > unixfile.txt
However, you cannot use tr to convert a document from Unix format to Windows.
awk
To use awk to convert a Windows file to Unix, enter:
awk '{ sub("r$", ""); print }' winfile.txt > unixfile.txt
To convert a Unix file to Windows, enter:
awk 'sub("$", "r")' unixfile.txt > winfile.txt
Older versions of awk do not include the sub function. In such cases, use the same command, but replace awk with gawk or nawk.
Perl
To convert a Windows text file to a Unix text file using Perl, enter:
perl -p -e 's/r$//' < winfile.txt > unixfile.txt
To convert from a Unix text file to a Windows text file, enter:
perl -p -e 's/n/rn/' < unixfile.txt > winfile.txt
You must use single quotation marks in either command line. This prevents your shell from trying to evaluate anything inside.
vi
In vi, you can remove carriage return ( ^M ) characters with the following command:
:1,$s/^M//g
To input the ^M character, press Ctrl-v, and then press Enter or return.
In vim, use :set to convert to Unix; use
ff=unix:set ff=dos to convert to Windows.
Related documents
Introduction
Files created in DOS/Windows use carriage return (r) and line feed (n) for line endings. However, files in Unix/Linux solely use line feed.
Therefore, when transferring a file from one system to another, make sure to convert the files.
In this tutorial, you will learn how to transfer files from DOS to Unix and vice-versa.
Converting Files on Linux
There are several ways you can transfer files to use the appropriate line endings. Convert the files using the:
dos2unixcommandunix2doscommandsedcommand (see Linux sed command tutorial)trcommand- Vi/Vim text editor (see Vim commands cheat sheet)
perlone-liner command
Option 1: Converting DOS to UNIX with dos2unix Command
The simplest way to convert line breaks in a text file is to use the dos2unix tool.
Install the tool by running the command:
sudo apt install dos2unixor:
sudo dnf install dos2unix
If you download a file created in DOS/Windows onto your Linux system, you can convert it using the dos2unix command:
dos2unix [file_name]The command converts the file without saving it in the original format. If you want to save the original file, add the -b attribute before the file name. Doing so creates a backup file under the same name and the .bak extension.
dos2unix -b [file_name]Option 2: Converting UNIX to DOS using the unix2dos Command
To convert a file created on a Linux system into the DOS format, run the command:
unix2dos [file_name]Alternatively, add the -b attribute to save the original file as backup:
unix2dos -b [file_name]Option 3: Using the sed Command
You can also use the sed (stream editor) command to remove the carriage return line endings. The command syntax to do so is:
sed 's/^M$//' [original_file_name]>[converted_file_name]Instead of just typing ^M, press Ctrl+V followed by Ctrl+M to enter the carriage return symbol. When using the sed command, specify the name of the DOS file [original_file_name] and how you want to name the converted file [converted_file_name].
To change the file format from Unix to DOS, use the command:
sed 's/$/^M/' [original_file_name]>[converted_file_name]Option 4: Using the tr Command
Another way to convert a file into the Unix format is to remove r line endings with the tr command. The tr command is a command line utility for translating or deleting characters.
Use the command in the following format:
tr -d 'r' < [original_file_name]>[converted_file_name]Option 5: Using the Vim Text Editor
You can also remove carriage return line endings from files in DOS format using the Vi/Vim text editor.
Open the file in Vi/Vim:
vi [file_name]Then press : and type in the following Vi/Vim command (making sure you type Ctrl+V then Ctrl+m instead of ^m):
:%s/^ m //g
Option 6: Using a Perl One Liner
Lastly, you can use a Perl one-liner command to delete all r line endings. Pearl on-liners are scripts that fit in a single line of code.
To replace all carriage return and line feed endings with just line feeds:
1. Open the file in the Vi/Vim text editor.
2. Press : to prompt the command line.
3. Type in the following command and press Enter:
perl -pi -e 's/rn/n/g' [file_name]
You should see the the changes in the file right away.
Example: Converting a Sample File from DOS to Unix Format
Let’s say you downloaded a file named sample-dos-file.
By opening the file using the Vim/Vi text editor, you would see that after each line ending there is ^M (a carriage return).
Another way to see the file uses both carriage return and line feed for line endings is to view the file in octal values.
To do so, run the command:
od -bc sample-dos-file.txtThe output displays the content of the file with its octal values, as in the image below. You can see that each line end is marked with the octal values 015 (r) and 012 (n).
Now, to convert the file and remove all carriage return endings, you would run the command:
dos2unix sample-dos-file
Open the same file in Vim/Vi. It doesn’t include any ^M symbols signaling the line ending.
To confirm the file is in Unix format, you can also open view the content in octal values. The output should display just the 012 values for n.
Conclusion
This article showed six ways to convert a DOS file into a Unix format and vice versa.
The simplest and recommended way is using the dos2unix command. If you cannot utilize dos2unix, you have five other options to choose from.
As a Linux administrator, you may have noticed some requests from developers to convert files from DOS format to Unix format, and vice versa.
This is because these files were created on a Windows system and copied to a Linux system for some reason.
It’s harmless, but some applications on the Linux system may not understand these new line of characters, so you need to convert them before using it.
DOS text files comes with carriage return (CR or r) and line feed (LF or n) pairs as their newline characters, whereas Unix text files only have line feed as their newline character.
There are many ways you can convert a DOS text file to a Unix format.
But I recommend using a special utility called dos2unix / unix2dos to convert text files between DOS and Unix formats.
- dos2unix: To convert a text files from the DOS format to the Unix format.
- unix2dos: To convert a text files from the Unix format to the DOS format.
- tr, awk and sed Command: These can be used for the same purpose
You can easily identify whether the file is DOS format or Unix format using the od (octal dump) command as shown below.
# od -bc windows.txt 0000000 125 156 151 170 040 151 163 040 141 040 146 162 145 145 040 157 U n i x i s a f r e e o 0000020 160 145 156 163 157 165 162 143 145 040 157 160 145 162 141 164 p e n s o u r c e o p e r a t 0000040 151 156 147 040 163 171 163 164 145 155 015 012 123 165 160 145 i n g s y s t e m r n S u p e 0000060 162 040 143 157 155 160 165 164 145 162 163 040 141 162 145 040 r c o m p u t e r s a r e 0000100 162 165 156 156 151 156 147 040 157 156 040 125 116 111 130 015 r u n n i n g o n U N I X r 0000120 012 071 065 045 040 157 146 040 167 145 142 163 151 164 145 163 n 9 5 % o f w e b s i t e s 0000140 040 141 162 145 040 162 165 156 156 151 156 147 040 157 156 040 a r e r u n n i n g o n 0000160 114 151 156 165 170 040 117 123 015 012 101 156 171 164 150 151 L i n u x O S r n A n y t h i 0000200 156 147 040 143 141 156 040 142 145 040 144 157 156 145 040 157 n g c a n b e d o n e o 0000220 156 040 114 151 156 165 170 015 012 n L i n u x r n 0000231
The above output clearly shows that this is a DOS format file because it contains the escape sequence rn.
At the same time, when you print the file output on your terminal you will get the output below.
# cat windows.txt Unix is a free opensource operating system Super computers are running on UNIX 95% of websites are running on Linux OS Anything can be done on Linux
How to Install dos2unix on Linux
dos2unix can be easily installed from the distribution official repository.
For RHEL/CentOS 6/7 systems, use the yum command to install dos2unix.
$ sudo yum install -y dos2unix
For RHEL/CentOS 8 and Fedora systems, use the dnf command to install dos2unix.
$ sudo yum install -y dos2unix
For Debian based systems, use the apt command or apt-get command to install dos2unix.
$ sudo apt-get update $ sudo apt-get install dos2unix
For openSUSE systems, use the zypper command to install dos2unix.
$ sudo zypper install -y dos2unix
1) How to Convert DOS file to UNIX format
The following command converts the “windows.txt” file from DOS to Unix format.
The modification of this file is to remove the “r” from each line of the file.
# dos2unix windows.txt dos2unix: converting file windows.txt to Unix format …
# cat windows.txt 0000000 125 156 151 170 040 151 163 040 141 040 146 162 145 145 040 157 U n i x i s a f r e e o 0000020 160 145 156 163 157 165 162 143 145 040 157 160 145 162 141 164 p e n s o u r c e o p e r a t 0000040 151 156 147 040 163 171 163 164 145 155 012 123 165 160 145 162 i n g s y s t e m n S u p e r 0000060 040 143 157 155 160 165 164 145 162 163 040 141 162 145 040 162 c o m p u t e r s a r e r 0000100 165 156 156 151 156 147 040 157 156 040 125 116 111 130 012 071 u n n i n g o n U N I X n 9 0000120 065 045 040 157 146 040 167 145 142 163 151 164 145 163 040 141 5 % o f w e b s i t e s a 0000140 162 145 040 162 165 156 156 151 156 147 040 157 156 040 114 151 r e r u n n i n g o n L i 0000160 156 165 170 040 117 123 012 101 156 171 164 150 151 156 147 040 n u x O S n A n y t h i n g 0000200 143 141 156 040 142 145 040 144 157 156 145 040 157 156 040 114 c a n b e d o n e o n L 0000220 151 156 165 170 012 i n u x n 0000225
The above command will overwrite the original file.
Use the following command if you want to keep the original file. This will save the converted output as a new file.
# dos2unix -n windows.txt unix.txt dos2unix: converting file windows.txt to file unix.txt in Unix format …
1a) How to Convert DOS file to UNIX format Using tr Command
As discussed at the beginning of the article, you can use the tr command to convert the DOS file to Unix format as shown below.
Syntax: tr -d 'r' < source_file > output_file
The below tr command converts the “windows.txt” DOS file to Unix format file “unix.txt”.
# tr -d 'r' < windows.txt >unix.txt
Make a note: You can’t use the tr command to convert a file from Unix format to Windows (DOS).
1b) How to Convert DOS file to UNIX format Using awk Command
Use the following awk command format to convert a DOS file to a Unix format.
Syntax: awk '{ sub("r$", ""); print }' source_file.txt > output_file.txt
The below awk command converts the “windows.txt” DOS file to Unix format file “unix.txt”.
# awk '{ sub("r$", ""); print }' windows.txt > unix.txt
2) How to Convert UNIX file to DOS format
When you convert a file from UNIX to DOS format, it will add a carriage return (CR or r) in each of the line.
# unix2dos unix.txt unix2dos: converting file unix.txt to DOS format …
This command will keep the original file.
# unix2dos -n unix.txt windows.txt unix2dos: converting file unix.txt to file windows.txt in DOS format …
2a) How to Convert UNIX file to DOS format Using awk Command
Use the following awk command format to convert UNIX file to DOS format.
Syntax: awk 'sub("$", "r")' source_file.txt > output_file.txt
The below awk command converts the “unix.txt” file to the DOS format file “windows.txt”.
# awk 'sub("$", "r")' unix.txt > windows.txt
I don’t want to use dos2unix tool. I want to use command in Ubuntu terminal to convert Windows file to Unix file. Is that possible. I have looked through other articles and tried those commands but none working. Anyone can help?
asked Apr 23, 2014 at 21:21
2
You can use tr to remove carriage returns like this:
tr -d 'r' < WindowsFile > UnixFile
You can use
cat -vet WindowsFile
to see if there are carriage returns in your file and they’ll show up as ^M
answered Apr 23, 2014 at 21:30
Mark SetchellMark Setchell
183k29 gold badges256 silver badges407 bronze badges
1
I want to use command in Ubuntu terminal to convert Windows file to Unix file.
dos2unix path/to/file/to/convert
I don’t want to use dos2unix tool.
alias notdos2unix=dos2unix
notdos2unix path/to/file/to/convert
answered Apr 24, 2014 at 1:24
ikegamiikegami
357k15 gold badges257 silver badges507 bronze badges
2
For Ubuntu(and Debian) you can use the tofrodos package.
sudo aptitude install tofrodos
fromdos file.txt # converts the file to UNIX line-endings
todos file.txt # converts the file to Windows line-endings
OSX (being a derivative of UNIX) should have the same line-endings as UNIX.
If you’re on Windows, you can find tofrodos binaries here.
If you’re on a Mac, you can use the brew package manager and install the tofrodos on OSX.
answered Apr 23, 2014 at 21:43
wsdookadrwsdookadr
2,5241 gold badge19 silver badges43 bronze badges
I have a PHP application with is located on Linux with multiple directories (and sub-directories) and many PHP, JS, HTML, CSS, etc files. Many of the files have Windows EOL control characters and I am also concerned that some might not be UTF-8 encoded but maybe ISO-8859-1, Windows-1252, etc. My desire is to convert all files to UTF-8 with LF only.
Looks like I might have a couple steps.
The dos2unix man provides this solution:
find . -name *.txt |xargs dos2unix
https://stackoverflow.com/a/11929475 provides this solution:
find . -type f -print0 | xargs -0 dos2unix
https://stackoverflow.com/a/7068241 provides this solution:
find ./ -type f -exec dos2unix {} ;
I recognize the first will only convert txt files which isn’t what I want but I can easily change to target all files using -type f. That being said, is one solution «better» than the other? If so, why? Is it possible to tell which files will be changed without changing them? When I finally change them, I don’t want the date to change, and intend to use dos2unix’s --keepdate flag. Should any other options be used?
Next, I will need to deal with encoding. https://stackoverflow.com/a/805474/1032531 recommends enca (or its sister command encov) and https://stackoverflow.com/a/64889/1032531 recommends iconv. It also seems like file might be applicable. Again, which one (or maybe something else all together) should be used? I installed enca and when executing enca --list languages, it lists several languages but not english (maybe choose «none»?), and I question is applicability. iconv was already installed, however, it does not have a man page (at least man iconv doesn’t result in one). How can this be used to recursively check and convert encoding?
Please confirm/correct my proposed solution or provide a complete solution.