Thursday 13 June 2019

Receiving a request with multiple file fields in NodeJS using multer

Multer is a handy module that enables the node API to upload files at ease. We can upload an array of files or files in separate fields or a single file using multer.

Let's see how one can upload files in received in separate fields.

In the following example, we are going to develop a request to upload a resume and a profile photo with field names 'resume' and 'profile_pic' respectively.

Receiving Request

var express = require('express');
var multer = require('multer');
var upload = multer({
    dest: 'upload_folder'
});
var app = express();

var upload_files = upload.fields([{
    name: 'resume',
    maxCount: 1
}, {
    name: 'profile_pic',
    maxCount: 8
}]);

app.post('/upload-profile', upload_files, 
     function(req, res, next) {
    console.dir(req.files); //prints the array of uploaded files

});

app.listen(parseInt(3200));


The following code is to be placed in your localhost(html folder):

Sending Request

<form action="http://localhost:3200/upload-profile" method="post" 
      enctype="multipart/form-data">
    <div>

        <label>Upload Resume</label> <input type="file" 
          name="resume" />

        <label>Upload profile pic</label> <input type="file" 
                    name="profile_pic" />

        <input type="submit" value="Submit">

    </div>

</form>


Hope this helps! Happy coding!

Thursday 25 April 2019

Uploading multiple files/array of files through curl in php

Many might have run in to issues while trying to upload multiple files or sending multipart form data to server through curl . Ideally, to send multiple files, one would save the files to an array variable and pass it through curl. But this would end up in error "Array to string conversion in /path/to/upload.php"

This means you are trying to send the files as mentioned in the below code:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<?php

$user = "admin";

$postField = array();

$postFields['user'] = $user; //postdata

$postFields['upload_file'] = array(

        'file[0]' => '@' . realpath('first-file.jpg'),
        'file[1]' => '@' . realpath('second-file.jpg')
    )

$headers = array("Content-Type" => "multipart/form-data");
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "http://serverdomain.com/upload/uploadMultipleFiles");
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl_handle, CURLOPT_POST, TRUE);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $postFields);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, TRUE);

$returned_data = curl_exec($curl_handle);
curl_close($curl_handle);
echo $returned_data ;
?> 


This is what should be the ideal code to send multiple files through php as per the manual, but pragmatically, it isn't right. You will not be able to send array of files through curl or technically, we can say, it won't support multi-dimensional arrays. Instead, you can send them as multiple CURL File objects
(PHP 5 >= 5.5.0, PHP 7) or single dimension array if you are using PHP <5. 

So the below code would help you send multiple files using CURL File object:




 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<?php

$user = "admin";

$postField = array();

$postFields['user'] = $user; //postdata

$postFields['upload_file_0'] = curl_file_create(
     realpath('first-file.jpg'), "mime", 'first-file.jpg' );

$postFields['upload_file_1'] = curl_file_create( 
     realpath('second-file.jpg'), "mime", 'second-file.jpg' );

$headers = array("Content-Type" => "multipart/form-data");
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "http://serverdomain.com/upload/uploadMultipleFiles");
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl_handle, CURLOPT_POST, TRUE);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $postFields);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, TRUE);

$returned_data = curl_exec($curl_handle);
curl_close($curl_handle);
echo $returned_data ;
?>



If your version of php doesn't support CURL File objects, you can declare the array as

$postFields['upload_file_0'] = '@' . realpath('first-file.jpg');//@ is mandatory to send files

$postFields['upload_file_1'] = '@' . realpath('first-file.jpg');

In case of more than two files or some 'n' number of files you can use loop:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<?php

$user = "admin";
$fileName = array();
$tempFileName = array();

 

$filecount = count($_FILES['upload_file']['name']);
for($i=0; $i<$filecount; $i++){
$fileName[] = $_FILES["upload_file"]['name'][$i];
$tempFileName[] = $_FILES["upload_file"]['tmp_name'][$i];
} 

 $postField = array();

$postFields['user'] = $user; //postdata

foreach ($tempFileName as $index => $file) {
  if (function_exists('curl_file_create')) { // For PHP 5.5+
    $file = curl_file_create($file, "mime", $fileName[$index]);
  } else {
    $file = '@' . realpath($file);
  }
  $postFields["upload_file_$index"] = $file;
}

$headers = array("Content-Type" => "multipart/form-data");
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL,  
"http://serverdomain.com/upload/uploadMultipleFiles");
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl_handle, CURLOPT_POST, TRUE);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $postFields);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, TRUE);

$returned_data = curl_exec($curl_handle);
curl_close($curl_handle);
echo $returned_data ;
?> 


Hope this helped you dig out a perfect solution!

Wednesday 20 March 2019

Migrating your wordpress from localhost to server or from one domain to other domain


Migrating your wordpress from localhost to server

To move the wordpress site from one host to the other host or server, you can directly make the changes in the MySQL database itself.
Login to your phpmyadmin or MySQL command line and use the following commands on your wordpress database:

1. 
  UPDATE wp_options
     SET option_value = 'http://new_domain_name.com'
     WHERE option_name = 'home';
2.
 UPDATE wp_options
    SET option_value = 'http://new_domain_name.com'
    WHERE option_name = 'siteurl';
3.
UPDATE wp_posts
    SET post_content = REPLACE(post_content,
    'http://old_domain_name.com', 'http://new_domain_name.com');
4.
UPDATE wp_posts
    SET guid = REPLACE(guid,'http://old_domain_name.com', 
     'http://new_domain_name.com');


To make sure there are no broken links, reset your permalinks by following the below steps:
  1. Log in to your WordPress admin panel.
  2. Click Settings.
  3. From the Settings menu, click Permalinks.
  4. Note which kind of Permalink you currently use.(for custom permalink, copy the current permalink structure and save it in a notepad for later use)
  5. Select any Permalink setting other than the one you currently use, and then click Save Changes.
  6. Select your original Permalink setting(paste the above copied permalink structure in custom permalink), and then click Save Changes.

Monday 18 March 2019

Finding last element in array - Check if current element is last element in Python arrays

When we are using loops in Python we might fumble onto a situation where we have to check if we reached the end of the array or simply say that we are iterating through each element in array and checking if the current element exists in a hash/dictionary/database and upon a match we still want to continue until the end of the array is reached.

So there are multiple ways of achieving this but I am going to give a simple solution that is easier to understand.

for suffix in suffixes:
        if(re.search(suffix+'$', my_string)):
            #do something
        elif(suffix == suffixes[-1]):
            print("reached end of array")

So the line that checks for the last element of array is

        elif(suffix == suffixes[-1]):

Similarily, to access for last but second element use suffixes[-2] and so on.

Hope its useful.

Thursday 14 March 2019

python capture regex groups in variables



#python script to capture regex matched groups into variables
import re

suffix = 'en'
word = 'children'

#print(word,suffix)

m = re.search(r'(.*)'+suffix + '$', word)

print(m.group(1))

Monday 11 March 2019

Split multi-paged PDF file in to individual PDF files

You might come across a situation where you have a PDF file with multiple pages and you'd like to split each page in to each PDF file. Sound pretty tedious task right?

It's just as simple as executing a single command. All you gotta do is, make sure you have pdftk (PDF Tool Kit) installed on your Linux machine.

If you do not have the pdftk installed, follow the below steps:

For(Ubuntu/Debian)


sudo apt update

sudo apt install pdftk 

 For(CentOS/Fedora), follow the instructions in the link below to install it:

https://www.pdflabs.com/docs/install-pdftk-on-redhat-or-centos/

Once installed, use the below command to split a single multi-paged PDF file to individual PDF files with each page

$pdftk your_multi-page_file.pdf burst

This will split each page from multi-page PDF file to each individual PDF file.


Similarly, to merge few selected PDF files in to one PDF file, use the following command:

$pdftk file1.pdf file2.pdf cat output mergedfile.pdf

If you want to merge all the PDF files in a specified directory, you may use the wild card character(*) as follows:

$pdftk *.pdf cat output mergedfile.pdf

'*' implies all the files with .pdf extension

You may find few more tips on dealing with PDF files here

Friday 8 March 2019

What is a hash or a dictionary in Python? Understanding hashes.

Hash tables or Dictionaries(as referred to in Python) are associative arrays. From Wikipedia associative arrays are defined as a collection of key, value pairs such that each key appears at most once in the hash table.

Question: Why can't we use arrays?

Answer: Because, when we use arrays its difficult to find an element in the array since the searching will loop through all the elements in the array until the element is found. This will compromise efficiency if the array is large in size. This problem is solved in hash table as elements can be accessed quickly without looping through the entire array.

Okay lets dig deeper...

I am going to explain this one with an example that we see daily. In this example we are going to store all members of a family relations and their names in a hash.

So the hash name is family_dict = {}

I am listing out all the elements I am going to store in it. It is going to contain wife, son, daughter, friend, father, mother....

All these relations have a name that we can call with. Now to build a hash we need keys and values. Identifying keys and values is the important thing because ultimately it will satisfy our need to use hash data structure.

In our hash we are going to store relations and their names. Before that one thing we all need to keep in mind while we build  a hash is that keys should be unique and values we will not care about them until there is really a need.

So our family hash needs unique things as keys. Names can't be unique as many people can be named with same name. This is as simple as that. Therefore, our hash is going to contain relations as keys and names as its corresponding values.

Now there will be a question what if our relations can also be same like when we have many brothers/sisters. Simply we are going to manipulate our keys are brother1, sister2 to make them unique. Enough of theory now and we will start our implementation.

family_dict = {
  "me": "Mr.x",
  "father": "Mr.y",
  "mother": "Mrs.z",
  "son":"kid1",
  "daughter1":"d1"
  "daughter2":"d2",
  "wife":"w1"
}

This seems simple. Each element mapped to its corresponding value. Now think of a situation where we might need to point same key to many values. For it lets assume Mr.x is a bit cheeky and has another wife w2. One just can't put two wives in same house. So in computers what happens is when you add same key with different value like "wife":"w2".  Our hash would store only one key that would be the one added last. That is previous keys are forgotten or overridden when same keys are added.

To solve this issue MR.x would compromise with his family and come to an agreement to put two of them in the same hut. But how do we do it here? No delay just scroll down.

family_dict = {
  "me": "Mr.x",
  "father": "Mr.y",
  "mother": "Mrs.z",
  "son":"kid1",
  "daughter1":"d1"
  "daughter2":"d2",
  "wife":"w1, w2"
}

If you observe clearly we just updated our hash key such that its value holds the previous one too. So in real time programming we should always check for existence of a key in hash if it already exists so as to make sure all our values being taken care of and none are overridden because of duplicate entry.

And this is how is is done. Now think of a hash which is going to store how many times a word occurs in the given text. For this we will store each word as a hash and its count as value. So everytime a word is revisited we will check for the value of the word and increment it by one.

    if word not in wordcount:
        wordcount[word] = 1
    else:
        wordcount[word] += 1

And finally we will print our hash:

#print the dictionary with sorted keys(words) and count as values
for k in sorted(wordcount):
    print (k, wordcount[k])

This has been a bit long post but thanks for coming here.
Bonus: Finding word frequency using Python

Wednesday 6 March 2019

How to open horizontal and vertical split mode to work with multiple files in VIM

Horizontal split mode

vim -o file1.txt file2.txt

Vertical split mode

vim -O file1.txt file2.txt

Now to switch between these files use Ctrl + W + W to toggle between these files.

By using these commands we can open n number of files.

PDF Splitter using PyPDF2 module of Python - Split PDF into multiple pages

Often when working with a large PDF we fumble upon of a need where we need each page of the PDF in one separate  PDF file.

So in this article we are exactly going to do this but not using any Linux command but using Python.

For that lets get our dependencies get installed. Just run the below command and you are all ready.

pip3 install pypdf2

Now open a editor and save the following code in to your editor.

from PyPDF2 import PdfFileWriter, PdfFileReader

inputpdf = PdfFileReader(open("largefile.pdf", "rb"))

for i in range(inputpdf.numPages):
    output = PdfFileWriter()
    output.addPage(inputpdf.getPage(i))
    with open("largefile-page%s.pdf" % i, "wb") as outputStream:
        output.write(outputStream)

Thanks to StackOverflow for this part. Thats it now you are ready to split a PDF file into multiple files with each file containing one page.

Thursday 28 February 2019

OpenNMT installation using PyTorch

Reference from OpenNMT Official Site

This manual will guide you through openNMT installation using PyTorch.

We assume that you have python3 already installed and 'pip3' the python3 package installer and your OS is assumed to be Linux(ideally Ubuntu)

Step1 Install PyTorch

pip3 install torch torchvision
If you are having python2 version then you may use the below command.
pip install torch torchvision
While this package downloads and installs you may have a cup of tea as this will take a while. Remember you need to have good internet connection as this package is about 582.5MB.

Step2 Clone the OpenNMT-py repository

git clone https://github.com/OpenNMT/OpenNMT-py
cd OpenNMT-py

Step3 Install required libraries

pip3 install -r requirements.txt

For python2 use

pip install -r requirements.txt
Thats it now you are ready to take off. To get familiarize about how to use openNMT follow the link

Monday 25 February 2019

Translation platform or Tools for Indian Languages

Translation platform often confused with Translation software is a tool/platform that aids a translator to use computer aided resources that are required to translate source language text into target language text.

While a translation software provides the possible Machine Translation of a source language text based on how its build where there is no human intervention. This machine translated output is most likely to be present in nonpunishable quality, meaning a human translator is needed to verify the machine generated translation and edit/review it so that the translation is perfect and publishable.



In today's world there are many translation platforms that support human translators to translate text at greater speeds and deliver high quality translations with publishable quality. European languages are very well supported with these kind of translation platforms. An Ideal translation platform should consist of the following features that can enable a translator to deliver high quality translation.4


  • Integrated with Multiple Machine Translations software that helps a translator to choose the most closely generated text from machine and then edit/review for further enhancements.
  • Availability of bilingual dictionaries/synonym dictionaries/Terminologies/Glossaries etc..
  • Translation Memory if available.
  • Transliteration tool.
  • Concordance search to search for some text in wide variety of available corpora.
  • Name Entity identification and Terminology identification.
  • Powerful target language spell checker.
  • Ability to add user's dictionary or existing Translation Memory.
One such tool that can help translators to deliver high quality publishable content is Transzaar.

Translation Memory Exchange(TMX)

Translation Memory Exchange or TMX is an xml file format for storing translation units for the exchange of translation memory data between computer-aided translation and localization tools with little or no loss of critical data.

<tmx version="1.4">
  <header
    creationtool="PyTool" creationtoolversion="1.01-023"
    datatype="PlainText" segtype="sentence"
    adminlang="en-us" srclang="en"
    o-tmf="ABCTransMem"/>
  <body>
    <tu>
      <tuv xml:lang="en">
        <seg>Hello world!</seg>
      </tuv>
      <tuv xml:lang="te">
        <seg>ప్రపంచానికి నమస్కారం</seg>
      </tuv>
    </tu>
  </body>
</tmx>

This is how a sample TMX file looks. Here I have given an example of English->Telugu translation Memory.

Translation Memory is useful in the following ways:

  • To recollect a past translation that has already been done and added to Translation Memory database.
  • Fuzzy search in Translation Memory helps to find out similar translations that can aid a translator.

Python Mysql connection and extracting sample data

In this tutorial we are going to learn how to connect to MYSQL using python.

First of all we need a config.py that will act as a configuration file that our python script will read. This configuration file consists information of MYSQL database username and password, database name and host where the MYSQL is at.

This is how config.py will look like

server = dict(
    #serverip = 'localhost',
    dbhost = 'localhost',
    dbname = 'userdb',
    dbuser = 'root',
    dbpassword = 'root123'
)

Then the actual python script that will connect using these parameters to our db and fetch results as needed.