Detection

• Detection Software
• Detection Products
• Method of Use

Detection Software

Various approaches are used to detect plagiarism at universities in Australia and New Zealand. The table below summarises the types of plagiarism detection software in use (if any) in respondent institutions. Software is being used, or considered for use, at most institutions, although seven currently do not use any software product.

It is clear that turnitin is the predominant software deployed, being used university-wide or in isolated faculties in 16 institutions, and being investigated by a further eight. Turnitin is the only plagiarism detection software product in wide use in Australasian universities, although it is clear that a range of other products is available.  The characteristics of these products will be compared in the following section. In addition, some institutions are developing their own software.

Types of plagiarism detection software in use in Australasian institutions.

Some of the products reported are not strictly plagiarism detection products.  Some (eg. Endnote and Power Researcher) are tools to facilitate correct referencing.  As the University of Waikato reports, Endnote enables students "to access our Library online catalogue and obtain all their bibliographical data very effectively that way, or from many other databases we have access to, and to seamlessly reference their assignments. It is an aid to good, efficient practice."

The University of Waikato also reports about Power Researcher:

"Students do their research for an assignment in Power Researcher; it tracks and automatically references material they access from the internet and enables them to reference it properly and also keeps archived copies just in case the material disappears from the internet when the lecturer goes back to check it."

"The broad view that was reached was that it was likely to be particularly effective at graduate level, as a way of facilitating good practice, but might be of rather limited benefit for lecturers teaching undergraduate papers."

Other tools, such as MOSS and YAP are specific to Computer Science. Finally, Google is used to search out plagiarism in some universities, but since this use is manual, it will not be considered as a detection product.

Across the sector, there was interest in detection software from the following disciplines:

• Business and management
• Computer Science, Informatics and Communication
• Law
• Art & Design
• Humanities

Some institutions were cautious about choosing plagiarism detection system. Curtin University of technology chose not to implement a plagiarism detection system because "a big part of the problem lies in the fact that too many assignments are designed in a way that lends itself to plagiarism".

The University of Waikato investigated plagiarism detection systems with an open mind, recognising that a plagiarism detection system was only one way of addressing a large and complex issue:

"We are actively interested in what software is available, but we are also sceptical; we don't immediately fall on anything that promises to be a software solution, because we collectively feel that there's a lot of hard yards to work to get widespread good practice and minimise plagiarism, and the software may serve particular purposes, but needs to be evaluated for its cost effectiveness quite carefully. There are limitations: Turnitin.com is quite clear at present about the databases that it can’t reach, and the subscription cheat sites that it can’t reach, but I also hear that it is going to improve its depth of access to these."

Detection Products

While Turnitin is clearly the most widely-used product, other similar products are available, and it is unclear why one product is so widely used.  Two possible reasons are that Turnitin has marketed its product more effectively, or that institutions have not carried out thorough feasibility studies.

The characteristics of the various products are described below.

Turnitin

Turnitin is a web-based product to which staff and students submit documents. The system then searches the Turnitin database, including all previous submissions from all sources, as well as searching web and bibliographic databases. Reports are returned with an indication of the amount of text matches found and any matches highlighted.  Human judgement is needed when reports are received, because properly cited quotes are also flagged as text matches. "it is still quite time consuming to use as academic judgements about use of references etc still need to be made." (University of SA)

One aspect of Turnitin is controversial, in that the system keeps copies of all work submitted for future comparison, which may be seen as violating students' intellectual property rights. Any new submission is compared to the entire Turnitin database. In this way, Turnitin detects not just plagiarism of published work, and collusion between students at a single institution, but also potential collusion between students from different institutions.

There are conflicting interpretations relating to the allocation of students' IP when using Turnitin.  ACODE recommends that individual universities should clarify the situation with the vendor and obtain independent legal advice if necessary

Some students object to assigning their Intellectual Property rights.  Universities also have concerns with Curtin University reporting a not uncommon view "unresolved legal issues about possible illegal use of a students’ work (by loading it into the Turnitin database without their permission)".

It seems that the likelihood of collusion between students from different institutions is relatively low, and the possibility (however remote) of student litigation might persuade some institutions to choose other products (such as MyDropBox and Scriptum) instead.  These products also detect plagiarism of published work, and collusion between students at a single institution is detected by searching separate, institution-specific databases.

A number of universities conducted trials of Turnitin before implementing it.  These are summarised in the table below.

The response from these trials, and from those who have subsequently adopted the product, has been largely positive.  As RMIT report:

"Feedback from staff to date on the use of Turnitin has been uniformly positive. All 37 staff members with approximately 1900 submissions in Semester 2 2004  have found it valuable and all want to continue using it."

Similarly, Massey report:

"Turnitin has proven effective for:

• Identifying blatant cheating
• Identifying collusion
• Revealing that many students do not understand the conventions, despite the resources they have available (paraphrasing, quoting, etc.), in the way that staff members expected. Although they have not deliberately cheated, their level of understanding and or use of academic conventions of writing is inadequate….
• Offering a deterrent to those who would otherwise have not used the correct writing conventions."

Other Products

A range of other plagiarism detection products is available on the market, and some institutions have developed their own solutions. The following table, derived from work at Murdoch University, compares the features of the commercial products for which data is available.

Both Scriptum and MyDropBox (Safe Assignment) seem to have similar functionality to Turnitin, and neither have the controversial Turnitin requirement for assignment of intellectual property.  No data is available about the effectiveness of the algorithms, and the breadth of data searched, of each product in detecting plagiarism.

James Cook University selected MyDropBox (Safe Assignment), following a thorough evaluation process.  There is no evidence of trialling or use of Scriptum in Australasia.

Open Source and Home-grown Products

Open source and home-grown products have been used at a number of institutions.

The Australian Defence Force Academy "is using a homegrown software that was developed over the last year. The prototype was a Perl script, developed by an academic, that scanned students’ work against the www and other students’ work. This has been developed over the last year into a user-friendly, web-based, multi-user interface that is accessible by all ADFA staff."

The Department of Information Systems at the University of Melbourne has developed their own plagiarism-detection software, which has been trialled in 5 faculties. Similarly, one faculty at the University of New South Wales has developed an "inhouse system, similar to a web-based search engine". However, UNSW felt that this might be "effective for particular disciplines, but may have limitations in terms of scalability and access to resources".

Data from Monash indicated that "A Monash PhD student, Kristztian Monostori, School of Computer Science and Software Engineering is reported (The Australian 20/2/01) to be developing a prototype system called "MatchDetectReveal (MDR) to detect plagiarism, in collaboration with Monash lecturers Arkady Zaslavsky, Heinz Schmidt and visiting professor Raphael Finkel."

Waikato University reported that "a member of staff has been involved in the development of another piece of software that meshes with MS Word and prompts and enables students to do good referencing as they paste text into assignments".

The Faculty of Informatics and Communication at Central Queensland University has established procedures using an open source plagiarism detection product integrated with their home-grown learning management system, WebFuse.

"The Faculty based its procedure on Culwin's and Lancaster's identification of a four-stage plagiarism detection process: collection, analysis, confirmation and investigation. We have two pieces of software we use for intra-corpal plagiarism detection

1. CopyFind http://plagiarism.phys.virginia.edu/software.html (Used for free text plagiarism detection, i.e. essays)
2. Moss http://www.cs.berkeley.edu/~aiken/moss.html (Used for computer programming assignments.)"

Miscellaneous

Little information is available about other products reported in use or under trial: ASSESS+COPICAT, EVE.

Methods of Use

Plagiarism Detection Systems such as Turnitin can be used either punitively or formatively.  While it is clear that plagiarism detection system is used to detect plagiarism and collusion where necessary, a number of institutions have also used plagiarism detection system in an educative sense, allowing students to submit drafts, and giving them a chance to revise work found to contain plagiarism.

"The University of Newcastle has a policy [in which] students are given the opportunity to check the report and amend their assessment if necessary."

"The standard way of using detection software is simply to punish, but the students should be encouraged to be involved." (University of Otago)

On the other hand, Massey University is currently focussing "on getting the punitive procedures right before we examine in depth the necessary changes required for a better educative approach".

The University of Canberra experimented with different ways to use Turnitin. The first approach was for students to submit their assignment to a support-staff member who would then submit it to Turnitin, and make recommendations and provide remedial support based on the resulting report. "This proved costly as it took too much time for the staff member to do the processing associated with Turnitin".

Secondly, they tried having students submit their assignment to Turnitin themselves, with the report going to the staff member who followed up with remedial support work with the students.

The third approach was to have students submit their assignment to Turnitin and receive the report themselves. The assignment plus report are submitted to their tutor, who then assesses whether the student needs help. If so, the tutor sends the report to a Turnitin support-staff member for recommendations.

RMIT University has adopted a deliberately systematic approach to its deployment of Turnitin, and it has developed a set of guidelines.

Guidelines for the use of Turnitin at RMIT

• The software must only be used by staff who have completed the Turnitin Application Form.
• There is full disclosure of the use of this software to all students in the courses in which Turnitin is used.
• All student content submitted to Turnitin is prefaced by a current attached RMIT Assessment Coversheet available from the Resources for Staff webpage. Students must submit a physical signed copy of this assessment coversheet before their work can be uploaded into Turnitin.
• Only Staff to upload content into the Turnitin site whilst implementation is refined. Student submission precluded to ensure equity of assessment practice.
• Upon detection of non-original work with the software, the responsible academic will inform the student that their assessable work must be original. Provide the student with information on how to avoid plagiarism, and require that the work be resubmitted within a limited timeframe.
• There is scope for variability in the number of assessment resubmissions acceptable within a given program, with the determination to be at the discretion of the Head of School, in conjunction with the relevant course and program coordinators. Any action on student plagiarism must be in full alignment with the RMIT Plagiarism Policy, with output from Turnitin available as evidence only, and not a singular determinant in the judgement of the plagiarism hearing.

The University of Newcastle has a policy that Turnitin must be used for all writing assessments. As part of this process: "When registering to attend the university, students must give consent for their work to be checked by an electronic plagiarism detection system. In addition, students must sign a cover sheet that goes on all assessments, stating that they have not plagiarised and re-informing them that their work may be checked in an electronic detection system."

Drawbacks of Plagiarism Detection Systems

A major drawback of plagiarism detection systems is that they are not plagiarism detection systems!  They are text matching systems. This means that text may actually be correctly quoted and cited, but reported as plagiarised, because the current systems do not recognise quoted text.  In addition, reference lists are also reported as matches, although they are clearly not plagiarised.  An article with no plagiarism could easily be reported as containing substantial amounts of matched text.

Because of this, text matching reports cannot be used as sole evidence of plagiarism, and academic judgement is required to confirm the extent and severity of any breaches. Some institutions reported that:

"[The] time required by a staff member to submit/assess the reports has also been a factor." (University of Canberra)

"While such software provides some level of deterrence for students and ease of locating sources for staff, it is still quite time consuming to use as academic judgements about use of references etc. still need to be made." (University of SA)

With the advent of the world-wide web, it became easier for students to plagiarise, and prior to the development of plagiarism detection systems, it was difficult, and very time consuming for staff to detect plagiarism.  For staff who have turned a blind eye to web-based plagiarism, plagiarism detection systems clearly add to the assessment workload.  On the other hand, for dedicated staff determined to detect plagiarism, for example by searching for text in Google, plagiarism detection systems are clearly more effective, and time efficient.

"Turnitin has been effective mainly as a labour-saving tool; it is used in conjunction with academic judgement." (Victoria University of Wellington)

"No matter what, they need to verify that work is not plagiarised. … Extra time is required by staff, but a class of 100 or so is reasonably easy when compared with some of the very large classes that are run in some institutions." (Australian Defence Force Academy).

While Plagiarism Detection Systems are not expensive compared to other educational software systems, the cost was still a factor for some institutions, especially those where initiatives were established at the faculty or school level.

A further drawback to plagiarism detection systems is the depth of access. While the range of sources used by each product is continually increasing, full coverage of the web may never be possible.  They also only search online articles, with the consequence that it is difficult to detect plagiarism of non-electronic academic journals and similar documents.

The pattern matching algorithms used by various products will also affect the effectiveness of plagiarism detection.  No evidence was uncovered in this work of a comparison of the effectiveness of algorithms.

Student perceptions about plagiarism detection approaches are relevant factors. In some cases, students feel that opportunities are being found to punish them, and they resent this. A dilemma has been recognised between the rights of a student to conscientiously object to having their work scrutinised, and students claiming this right to avoid detection.  The use of Turnitin compounds this problem, because this company may be seen to claim rights to the student’s intellectual property as well.  This issue does not seem to have been addressed widely, to date.

The University of Sydney reports "The Teaching & Learning Committee has taken a restrictive approach to the use of Turnitin, as the use of any type of plagiarism-detection engine must give students the right to opt out. This restriction is being assessed by the Committee. Given this restriction, academic staff have been cautious about the use of plagiarism detection tools. However, the Faculty's policy is to educate students on best practice rather than to detect and punish students."

A further drawback of the available products is that they focus on text-based plagiarism and collusion.  Some institutions are using products like MOSS and YAP to detect similarities in computer program code. However, several institutions recognised that plagiarism of diagrams and pictures is a potential issue, and there seems to be a gap in the market for products which detect this type of plagiarism. The only product identified was a program called ArtSmart, currently being developed by the University of Melbourne.

Staff reluctance and possible lack of skills was also an issue at some institutions. One of the reasons that Curtin University chose not to implement a plagiarism detection system, was that

 "…even if we have it, there is no guarantee that staff will use it."

Similarly, the Australian Defence Force Academy reported that

"Some staff have been reluctant to use the plagiarism detection system because of the technology barrier. Our approach is to make the process (of submission, evaluation and report checking) as seamless as possible. It is necessary to engage with staff and encourage them to make sure they participate in the whole academic conduct process. Some staff may also be reluctant to use the EPDS because they are fearful of what it will reveal."

Beliefs about student integrity

There were divergent views between institutions about the integrity of students.

Some institutions had the view that some students would cheat if they had the opportunity. Auckland University of Technology reported that:

"Turnitin is used for both final assignment submission as well as submission of the draft (ie it is used both summatively and formatively). However, the Business faculty’s experience has been that the formative aspect of the draft submission can be a double-edged sword. Students who have inadvertently plagiarised are able to learn from their mistakes and correct the citation etc. However, other students use the draft submissions strategically to "see what they can get away with" ie they change odd words till Turnitin no longer recognises the plagiarism."

Such institutions do not favour allowing students to submit drafts of their work into the detection software because that will aid them to work out less detectable ways to plagiarise:

"Students can avoid detection with a ts of their work into the detection software because the feedback they receive will teach them about correct referencing. The University of QueenslaTurnitin draft submission.  Simply change a few words and retest. Turnitin tests strings of 8 words. It accesses only certain pages, database, journals on the internet." (Australian Defence Force Academy)

Other institutions prefer to trust in the integrity of their students, focussing on educative rather than punitive use of a plagiarism detection system. These institutions encourage students to submit drafnd's "approach to managing plagiarism is to foster high ideals in ethics rather than to "catch" students who are cheating". Murdoch University sees a close alignment between this approach and one of its stated graduate attributes:

"An awareness of and sensitivity to ethics and ethical standards on interpersonal and social levels, and within a field of study and/or profession."

Similarly, James Cook University sees benefits in the submission of drafts:

"The software we use allows a student to put in a draft. If too heavily matched, it can be sent back to [the] student for a second try so that they can then get their citations correct. They can rethink their whole approach to the paper. Some students may try swapping words more cunningly. You can't really guard against that level of deviousness, you're really trying to teach students how to think on their own and so I think the draft capacity of SafeAssignment promotes that point of view. It gives them a second chance, shows that teachers are there to help them rather than catch them out. It helps them to think independently."

Cultural issues

There was some evidence that plagiarism is more common in students from non-Anglo-Saxon cultures. International students in their early years were identified as 'at risk' of plagiarism, and some institutions have mechanisms in place to address these issues:

"Plagiarism may occur because of cultural issues, such as misunderstanding academic conventions in a particular discipline or a level of English language skills that prevents adequate expression  ... Whatever the reasons there are actions for all members of the university community that will minimise cases of plagiarism. For this reason the University provides a focus on appropriate learning, teaching and assessment strategies." (University of Adelaide)

"Since 1996, a rapid increase in the number of international students with experiences of different educational cultures and with different expectations about appropriate student behaviour led to increased frequency of plagiarism and problems with referencing. CQU responded by a comprehensive review of practice that included:

• Revision of policies and procedures on plagiarism
• Work with the academic community to develop a clear minimum set of referencing standards for undergraduate students
• Development of educational resources to explain referencing requirements to learners and academics
• Revision of assessment policies and guidelines for students and academics." (Central Queensland University)

In a plagiarism detection system trial held at Murdoch University, the only substantial plagiarism detected was in a cohort of students at an offshore partner campus. There was little plagiarism among students studying the same unit in Australia. This incident may not indicate that plagiarism is a problem only offshore.  Students at Murdoch may also have plagiarised if their lecturer had not emphasised the issues of plagiarism and the likelihood to her students.

Feedback from the offshore campus was that they were not adequately warned about the plagiarism detection system trial. Nevertheless, the hierarchy at the offshore institution responded positively to the issue, and have started advising and educating students about plagiarism, and are using the availability of a plagiarism detection system as a quality assurance and marketing tool.

Summary

Some issues arose about the ways in which plagiarism detection software could be used, including:

• The need for academic judgement in the use of any software;
• Whether to use software for formative or punitive purposes;
• Staff/ institutional beliefs about student integrity – whether students should be viewed as intrinsically ethical, or whether they would cheat at any available opportunity;
• The cultural issues associated with academic integrity.