Sunday 6 November 2016

A simple perl script to make a file content unique

This simple perl script does the job of sort -u (of course not in windows)

Script:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
#!/usr/bin/perl

use strict;

use warnings;


#File reading code

open(my $in,"<:utf8",$ARGV[0]) or die "Cannot open $ARGV[0]:$!\n";

my @in =<$in>;

close($in);


#declare hash

my %hash=();


#store content in hash

foreach my $in (@in) {

 chomp($in);

 $hash{$in} = 1;

}


#Finally print the hash

foreach my $keys(sort keys %hash){

 print "$keys\n";

}
How to run:

Lets assume that our script is saved as my_unique.pl 


$ perl my_unique.pl <filename>

* Without angular brackets.

Explanation:
Here in Perl hash the duplicate keys are overridden which means only single instance of each line in the file is stored in the hash. Remember that while storing the lines in hash each single line is treated as key so as to remove duplicates. 

Note: Always use strict and use warnings when dealing with Perl code so as to write bug free code.

No comments:

Post a Comment