代写COMP34711、代写Python，c++编程

日期：2021-12-16 10:33

COMP34711 Natural Language Processing

Coursework 2, Nov 2021

You are provided with the product review corpus. Check the README file and observe the content,

format and structure of the corpus. You are asked to design and evaluate solutions for two NLP

tasks using this corpus. You are only free to use functions that are available in the NLTK

framework and machine learning libraries specified in the instruction, e.g., Weka, scikit-learn,

PyTorch, Keras, TensorFlow, to implement your design. Overall, this coursework is marked on the

basis of

? rigorous experimentation,

? knowledge displayed in report,

? independent problem-solving skill,

? self-learning ability,

? how informative your analysis is,

? language and ease of reading of the report,

? code quality based on correctness and readability (which includes comments).

You should solve all the tasks on your own. You are not permitted to collaborate with other

students on this coursework. In lab support sessions, you can ask TAs to explain knowledge

taught in the lecture or seek advice on how to use a natural language processing or machine

learning library. But you are not permitted to ask TAs to help with the solution design, or to check

the correctness of your solution.

Your submission should include both code and report. About your code, provide comments when

you see fit and your code will be marked based on both correctness and readability (which

includes comments). About your report, use Arial Font 11. Your main report should be no more

than 3 pages, including up to 2 pages for Task 1 while up to 1 page for Task 2. If needed, you can

include additionally up to 2 pages of screenshots (e.g., of your results) as an Appendix of your

report.

Task 1: Distributional Semantics (15 marks)

The following experiment is designed to evaluate the performance of a distributional semantic

approach.

? Step 1: Clean and pre-process all reviews in your text corpus as you see fit. Choose the top 50

most frequently occurred words (after removing the stop words) as the target words. You are

free to use functions that are available in the NLTK framework to help your text pre-processing.

? Step 2: For each of the 50 target words, uniformly sample half of its occurrences in the corpus

and substitute these with a made-up reverse words, e.g., half of the occurrences of "canon'' will

be transformed into "nonac". Refer to these 50 new words as pseudowords.

? Step 3: Construct a d-dimensional feature vector to characterise each of the 50 target words

and 50 pseudowords (N=50+50=100) using a distributional semantic approach (more detailed

requirements on this are provided later). Store your obtained feature vectors in a 100′d matrix

X.

? Step 4: Take the feature matrix X as the input, and apply a clustering algorithm to cluster the

100 words into 50 clusters. You are free to use eixsing clustering algorithm implementation as

2

you see fit. For instance, clustering modules (https://www.nltk.org/api/nltk.cluster.html) from

NLTK, machine-learning framework for clustering from Weka

(http://www.cs.waikato.ac.nz/ml/weka/) and scikit-learn (https://scikitlearn.org/stable/modules/clustering.html#clustering).

? Step 5: For each pair of the target word and its corresponding pseudoword, if these two are

grouped into the same cluster, it is defined as a correct pair. Among the 50 pairs, check the

percentage of the correct pairs, denoted by p.

? Step 6: Repeat this whole process multiple times, e.g., 5-10, and calculate the mean and

standard deviation of the obtained percentages p.

Applying what you have learned on lexical processing and distributional semantics, you should

come up with 2 different approaches for constructing the distributional semantic representations.

For instance, they can differ in ways of constructing the dictionary (e.g., stems vs. words) and of

extracting the context features, or differ in the approach principles. You should aim at achieving a

good clustering performance and understanding the reason behind.

Here are the requirements of the 2 approaches:

? They should differ significantly. For instance, the same context feature extraction approach

with different window sizes is considered as one approach.

? They should include one sparse approach and one dense approach.

? They should be evaluated and compared thoroughly, e.g., their performance, and effect of

their hyperparameter setting.

1. Submission Instruction

Your implementation should be well-structured, defining a function for each step and executing the

functions in a main file.

You should submit the implementation and evaluation of your 2 approaches as 2 separate Jupyter

notebook files, named as “Task1_Approach1”, “Task1_Approach2”. The TA will run each file

separately during marking.

You should prepare a report (up to 2 pages) containing two sections:

? Methods: Explanation of your text cleaning and pre-processing steps, as well as the 2

approaches for constructing the distributional semantic representations.

? Result Analysis: Analyse and discuss the obtained clustering results for each approach.

You should discuss hyperparameter relevant issues if your approach requires any

hyperparameter setting, e.g., setting context window size, determining feature

dimensionality d for a dense approach, etc.

2. Mark Allocation

Marks are allocated as below:

? 1 mark for text cleaning, pre-processing, target words selection, pseudo words

construction.

? 10 marks for implementation, description, and result analysis of the 2 approaches, where 5

marks for each approach.

3

? 2 marks for clustering performance award, which means to achieve a satisfactory clustering

performance exceeding a percentage threshold by at least one approach and for explaining

the reason behind your success. This percentage threshold will only be released to you

after the marking.

? 2 marks for design novelty that can be either an improvement of what has been taught or a

new reasonable approach not taught in the “Distributional Semantics” Chapter. You need to

highlight in the report what the novelty is, if to gain these marks.

Task 2: Neural Network for Classifying Product Reviews (10 marks)

The product review corpus contains reviews scored as positive and negative opinions. Pre-process

your text, prepare the review examples for training and evaluation. Implement, train and evaluate a

neural network that can classify an input review to either a positive or a negative class. You are

free to choose any neural network/deep learning technique taught in the Chapter “Deep Learning

for NLP”, e.g., multi-layer perceptron, LSTM, bi-directional LSTM, etc. You should evaluate your

classifier’s classification accuracy using 5-fold cross validation (CV). You are free to use PyTorch,

Keras or TensorFlow library.

1. Submission Instruction

Your implementation should be well-structured with comments. You should submit the

implementation and its evaluation as a single file, named as “Task2”.

Prepare a report (up to 1 page) containing 2 short sections:

? Method: Explanation of your classification model design and training.

? Experiment and Result Analysis: Describe your experiment and evaluation approach. You

should discuss hyperparameter relevant issues if your approach requires any

hyperparameter setting. Report and analyse classification accuracy.

2. Mark Allocation

Marks are allocated as below:

? 2 marks for text cleaning, pre-processing, and preparing the input data for the classifier.

? 7 marks for implementation, classification accuracy evaluation by 5-fold cross validation,

method description, and result analysis of the classifier.

? 1 mark for classification accuracy award, which is to achieve a satisfactory classification

accuracy exceeding an accuracy threshold. This threshold will only be released to you after

the marking.

-----------------------------

Submission Checklist

A .zip file named as “34711-Cwk-S-DeepLearning” containing

? Three code files: Task1_Approach1, Task1_Approach2, Task2.

? One .pdf file, combining reports for both Task 1 and Task 2.