Project Part 1 - sample output

[Sushovan De <sushovan@asu.edu>]

Some of you wanted to verify whether the output of your code was correct. Here are some sample outputs that I got, and my assumptions. If you used different assumptions, you would get different results, but it doesn't mean that your results are wrong.

Assumptions:

TF value is not normalized

IDF value is not normalized

in the TF-based similarity, all query words are given equal weight

in the TF-IDF based similarity, the query words are weight by IDF alone, not (0.5 + 0.5*tf)*idf [slide 62]

TF-based similarity

query> fall semester

[5]result3/www.asu.edu/aad/catalog-seminar/calendars/course-related-deadlines.html: 0.41959067914834447

[22932]result3/www.asu.edu/registrar/general/semcal.html: 0.3900947488027469

[22926]result3/www.asu.edu/registrar/general/ProjectedDatesforFutureSemesters.html: 0.3031695312954162

[22978]result3/www.asu.edu/sbs/paying_semester_selection.html: 0.28127197523150593

[3765]result3/www.asu.edu/fs/advantage/com/308-12.htm: 0.27980420556019375

[14669]result3/www.asu.edu/lib/archives/archive.htm: 0.2644429426739725

[22980]result3/www.asu.edu/sbs/tuition_semester_selection.html: 0.25993762245501817

[871]result3/www.asu.edu/aad/manuals/usi/usi301-02.html: 0.2434322477800738

[991]result3/www.asu.edu/admissions/nondegree/screen5.html: 0.23947373603569988

[1894]result3/www.asu.edu/clas/reesc/cli/yerevan2.htm: 0.23416772415725648

query> transcripts

[22956]result3/www.asu.edu/registrar/transcripts/index.html: 0.3460520089727601

[1055]result3/www.asu.edu/admissions/steps/transcripts.html: 0.26490647141300877

[3927]result3/www.asu.edu/graduate/admissions/admissions_checklist.htm: 0.1346153846153846

[22124]result3/www.asu.edu/provost/articulation/atp/atp_instructions_INT.htm: 0.1125087900926024

[22123]result3/www.asu.edu/provost/articulation/atp/atp_instructions.htm: 0.1125087900926024

[4145]result3/www.asu.edu/graduate/generalinfo/DeMund/index.html: 0.10283867552865913

[3930]result3/www.asu.edu/graduate/admissions/domestic.html: 0.10173145278829236

[16633]result3/www.asu.edu/lib/archives/shema/shema.htm: 0.09531918345647832

[3871]result3/www.asu.edu/graduate/SEM/admissions/requirements.htm: 0.08987062325802277

[22928]result3/www.asu.edu/registrar/general/contacts.html: 0.08728715609439695

query> admissions

[1075]result3/www.asu.edu/admissions/visitcampus/tempecampus/appointment.html: 0.457495710997814

[3759]result3/www.asu.edu/forms/index.html: 0.4375949744936837

[3758]result3/www.asu.edu/forms/forms_adm.htm: 0.41178049395232924

[964]result3/www.asu.edu/admissions/international/index.html: 0.35909242322980395

[962]result3/www.asu.edu/admissions/international/expenses.html: 0.33409177193234335

[961]result3/www.asu.edu/admissions/international/definition.html: 0.3274461888074916

[936]result3/www.asu.edu/admissions/contact/familymember.html: 0.31983816366946766

[22446]result3/www.asu.edu/provost/committees/FAS.html: 0.30983866769659335

[935]result3/www.asu.edu/admissions/contact/counselor.html: 0.2950148263727627

[937]result3/www.asu.edu/admissions/contact/freshman.html: 0.28966206101082875

TF-IDF based similarity:

query> fall semester

[22932]result3/www.asu.edu/registrar/general/semcal.html: 0.4359373503377644

[5]result3/www.asu.edu/aad/catalog-seminar/calendars/course-related-deadlines.html: 0.33526131786429103

[14669]result3/www.asu.edu/lib/archives/archive.htm: 0.2871305669297931

[991]result3/www.asu.edu/admissions/nondegree/screen5.html: 0.27532125556740966

[22978]result3/www.asu.edu/sbs/paying_semester_selection.html: 0.26913921652871753

[22980]result3/www.asu.edu/sbs/tuition_semester_selection.html: 0.2601997953589357

[2219]result3/www.asu.edu/data_admin/data_warehouse-registered_student.html: 0.24907963439080882

[22743]result3/www.asu.edu/provost/spiada/data_admin/data_warehouse-registered_student.html: 0.24907963439080882

[23071]result3/www.asu.edu/spiada/data_admin/data_warehouse-registered_student.html: 0.24907963439080882

[3765]result3/www.asu.edu/fs/advantage/com/308-12.htm: 0.24906102399484592

query> admissions

[3758]result3/www.asu.edu/forms/forms_adm.htm: 0.5744080507440753

[1075]result3/www.asu.edu/admissions/visitcampus/tempecampus/appointment.html: 0.5690209713731555

[938]result3/www.asu.edu/admissions/contact/index.html: 0.5031740605651273

[964]result3/www.asu.edu/admissions/international/index.html: 0.43059540671762864

[936]result3/www.asu.edu/admissions/contact/familymember.html: 0.4172350134375292

[963]result3/www.asu.edu/admissions/international/faq.html: 0.40879880279063435

[961]result3/www.asu.edu/admissions/international/definition.html: 0.3951636676211233

[962]result3/www.asu.edu/admissions/international/expenses.html: 0.38631455298928125

[3759]result3/www.asu.edu/forms/index.html: 0.38614177431456176

[937]result3/www.asu.edu/admissions/contact/freshman.html: 0.38147065345148296

query> transcripts

[22956]result3/www.asu.edu/registrar/transcripts/index.html: 0.5123337680008732

[1055]result3/www.asu.edu/admissions/steps/transcripts.html: 0.47577601236762335

[3927]result3/www.asu.edu/graduate/admissions/admissions_checklist.htm: 0.2195190097057037

[3930]result3/www.asu.edu/graduate/admissions/domestic.html: 0.18180264245956912

[22124]result3/www.asu.edu/provost/articulation/atp/atp_instructions_INT.htm: 0.16624013235632593

[22123]result3/www.asu.edu/provost/articulation/atp/atp_instructions.htm: 0.16624013235632593

[3871]result3/www.asu.edu/graduate/SEM/admissions/requirements.htm: 0.15646994303347006

[1051]result3/www.asu.edu/admissions/steps/printsteps.html: 0.1554359618428368

[3834]result3/www.asu.edu/graduate/SEM/8admiss.htm: 0.14754024695739043

[999]result3/www.asu.edu/admissions/orientation/downtownphoenixorientation/downtownpreorientation.html: 0.1425772012274932

Thanks and Regards,
Sushovan De