Submission Deadline
Marks and Feedback
Before 10am on:
Tue 17/12/2019 Fri 20/12/2019 (if required an extension could be granted by Student Mitigation Team)
20 working days after deadline (L4, 5 and 7)
15 working days after deadline (L6)
10 working days after deadline (block delivery)
Unit title & code
CIS111-6: Intelligent Systems and Data Mining
Assignment number and title
Assignment 1: Data Mining Solutions for Direct Marketing Campaign
Assessment type
WR
Weighting of assessment
50%
Unit learning outcomes
Analyse a Data Mining technique capable of supporting practitioners to make reliable decisions which require predictive modelling, for example, in a Business scenario
Demonstrate results of using an efficient technique which is capable of finding a solution to a given predictive problem represented by a data set
Evaluate the accuracy of the technique in terms of differences between the predicted values and the given data
What am I required to do in this assignment?
1. Task
Students will develop a DM solution for saving cost of a direct marketing campaign by reducing false positive (wasted call) and false negative (missed customer) decisions. Working on this assignment, students can consider the following scenario.
A Bank has decided to save the cost of a direct marketing campaign based on phone calls offering a product to a client. A cost efficient solution is expected to support the campaign with predictions for a given client profile whether the client subscribes to the product or not. A startup company wants to develop an innovative DM technology which will be competitive on the market. The Manager will interview and hire Data Analysts. The team will analyse the existing technologies to design a DM solution winning the competition. A team Manager will choose the best solution for the market competition in terms of cost efficiency. The evaluation of the developed solutions will be made on the test data. The costs will be defined for both the false positive and false negative predictions.
Examples of cost-efficient DM solutions for direct marketing are provided on the UCI Machine Learning repository describing a Bank Marketing problem.
Students will apply for one of roles: (i) group manager, (ii) group member, or will work individually. The group manager will arrange comparison and ranking of solutions designed in a group, and will have additional 5 points. Each student will run individual experiments to find an efficient solution and describe differences in experimental results.
2. Method and Technology
To design a solution, students will use Data Mining techniques such as Decision Trees and Artificial Neural Networks. Examples of solutions will be provided in R Scripting using (i) a Cloud technology CoCalc or (ii) an advanced development suit RStudio free for students.
3. Data
The Assignment 1 Bank Marketing data set is available as a csv file.
4. Report submission and report template
Each solution will be evaluated in terms of the costs of false decisions made on the validation data. Reports will be submitted via BREO. Reports can be prepared with a template. BREO similarity level of reports must be < 20%.
Is there a size limit?
2000 words on average
What do I need to do to pass? (Threshold Expectations from UIF)
Apply Decision Tree technique to solve the Bank Marketing task presented by a set of customer profiles
Analyse problems which are required to be resolved in order to develop a solution providing a high prediction accuracy on a given data set.
How do I produce high quality work that merits a good grade?
Identify a set of parameters which are required to be adjusted within DM techniques in order to optimise the solution in terms of prediction accuracy
Explain how the parameters of a DM technique influence the prediction accuracy
Run experiments in order to verify the solution designed on the given data set
Analyse and compare the results of the experiments in a group and with the known from the literature.
How does assignment relate to what we are doing in scheduled sessions?
Data Mining techniques and use cases developed in R will be considered during lectures and tutorials.
How will my assignment be marked?
Your assignment be marked according to the threshold expectations and the criteria on the following page.
You can use them to evaluate your own work and estimate your grade before you submit.
#
Weight, %
Lower 2nd – 50-59%
Upper 2nd – 60-69%
1st Class – 70%+
1
Analysis
(20)
Fair analysis of the basic approaches
Relatively good analysis of the relevant literature, mainly covering the state-of-art
Excellent analysis of the relevant literature, fully covering the state-of-art
2
Design
(50)
Fair design of a basic solution providing a reasonable performance within a single set of parameters
Design of a solution providing a fair performance in a series of experiments with different sets of parameters
Design of a solution providing a performance, competitive to known from the literature, in a series of experiments with different sets of parameters
3
Conclusion (30)
Fair conclusion on the experimental results obtained within a single set of parameters
Conclusion on and comparison of the experimental results obtained within two different sets of parameters
Conclusion on and comparison of the experimental results obtained within multiple sets of parameters, demonstrating a solution which provides a competitive performance
