Import Libraries¶

In [12]:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
import pandas as pd

Input text and labels¶

In [7]:
texts = [
    "a wonderful film with a moving story",      # positive
    "great acting and a lovely script",          # positive
    "i loved every minute, brilliant",           # positive
    "an excellent and enjoyable film",           # positive
    "a dull film with a weak plot",              # negative
    "terrible acting and a boring script",       # negative
    "i hated it, a complete waste of time",      # negative
    "awful, slow and disappointing",             # negative
    "the film is two hours long",                # neutral
    "it was released in the summer",             # neutral
    "the story follows three friends",           # neutral
    "the director also wrote the script",        # neutral
]
labels = ["pos"]*4 + ["neg"]*4 + ["neu"]*4
print(labels)

['pos', 'pos', 'pos', 'pos', 'neg', 'neg', 'neg', 'neg', 'neu', 'neu', 'neu', 'neu']
In [5]:
vec=TfidfVectorizer()  # Creates a tool that converts tes
In [14]:
X = vec.fit_transform(texts)

fit: reads all 12 reviews, builds the vocabulary (46 distinct words in this dataset), and works out the IDF of each word, as on the TF-IDF slide.¶

transform: uses that vocabulary and those IDF values to convert each review into a row of numbers.¶

In [15]:
df = pd.DataFrame(X.toarray(), columns=vec.get_feature_names_out()).round(2)
print(df)

    acting  also   an   and  awful  boring  brilliant  complete  director  \
0     0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.00      0.00   
1     0.44  0.00  0.0  0.35   0.00    0.00        0.0      0.00      0.00   
2     0.00  0.00  0.0  0.00   0.00    0.00        0.5      0.00      0.00   
3     0.00  0.00  0.5  0.34   0.00    0.00        0.0      0.00      0.00   
4     0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.00      0.00   
5     0.44  0.00  0.0  0.35   0.00    0.51        0.0      0.00      0.00   
6     0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.42      0.00   
7     0.00  0.00  0.0  0.37   0.54    0.00        0.0      0.00      0.00   
8     0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.00      0.00   
9     0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.00      0.00   
10    0.00  0.00  0.0  0.00   0.00    0.00        0.0      0.00      0.00   
11    0.00  0.43  0.0  0.00   0.00    0.00        0.0      0.00      0.43   

    disappointing  ...   the  three  time   two   was  waste  weak  with  \
0            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.43   
1            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.00   
2            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.00   
3            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.00   
4            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.49  0.42   
5            0.00  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.00   
6            0.00  ...  0.00   0.00  0.42  0.00  0.00   0.42  0.00  0.00   
7            0.54  ...  0.00   0.00  0.00  0.00  0.00   0.00  0.00  0.00   
8            0.00  ...  0.31   0.00  0.00  0.45  0.00   0.00  0.00  0.00   
9            0.00  ...  0.30   0.00  0.00  0.00  0.44   0.00  0.00  0.00   
10           0.00  ...  0.33   0.49  0.00  0.00  0.00   0.00  0.00  0.00   
11           0.00  ...  0.58   0.00  0.00  0.00  0.00   0.00  0.00  0.00   

    wonderful  wrote  
0         0.5   0.00  
1         0.0   0.00  
2         0.0   0.00  
3         0.0   0.00  
4         0.0   0.00  
5         0.0   0.00  
6         0.0   0.00  
7         0.0   0.00  
8         0.0   0.00  
9         0.0   0.00  
10        0.0   0.00  
11        0.0   0.43  

[12 rows x 46 columns]
In [9]:
clf = LogisticRegression(max_iter=1000).fit(X, labels)

Now Testing on unseen data¶

In [10]:
tests = ["a brilliant and moving film",
         "a boring and weak plot",
         "not a brilliant film",
         "a film that is not boring"]
for t in tests:
    print(t, "->", clf.predict(vec.transform([t]))[0])
print("vocabulary size:", len(vec.vocabulary_))

a brilliant and moving film -> pos
a boring and weak plot -> neg
not a brilliant film -> pos
a film that is not boring -> neg
vocabulary size: 46
In [ ]: