Logic of hypothesis testing and types of errors

Photo by on

You might have come across innumerable claims and statements involving numbers, especially in marketing campaigns and ads. “9 out of 10 doctors recommend Colgate toothpaste”, or “Dettol kills 99.9% of bacteria” are classic examples of numerical claims. The statistical validity of such statements regarding a parameter can be tested if…

Text cleaning as part of preprocessing for Text Analytics

Common punctuation marks seen in text. (Source: )

Removal of punctuation is a necessary step in cleaning the text data before performing text analytics. Python offers numerous ways to deal with punctuation. Below given is a simple implementation using ‘re’ and ‘string’ modules.

import re
import string

The punctuation attribute of ‘string’ module is used as the reference list to look for all possible punctuation in the text data. Then, substitute function from ‘re’ is used to replace all punctuation from the target string or text data.

s = "A@p,p!!le#"
punctuation = '['+string.punctuation+']'
re.sub(punctuation,'',s)

The output of the last line above is:

'Apple'

Thank you! Stay tuned for more interesting things you can do with Python!

Anjana K V

Data Science Professional | 7+ years of experience in data science & analytics across various domains — retail, insurance, finance and digital marketing

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store