2005 NYD FAQ |
(Note: except the on-demand Reputation Rankings — no student data
Q1. Where do the rankings come from?
All StudentsReview Rankings
are derived completely from student surveys. We have absolutely
no personal input or investment in the outcome of the
rankings, and do not adjust them to make any school
look good. They are completely and totally generated
from the student surveys. We set the metrics for
what constitutes a “best education”, or a “Top Creative School”,
and allowed the student opinion to speak for itself.
Q2. What is the point of your rankings and
how are they different?
Rankings need to have an explicity purpose
that is insightful to its readers. It is not
merely enough to say the “Top 50” , but rather
to describe what constitutes a Top 50 university, why the
constitution should be that way, and then what schools match.
It is only when the purpose is clearly described
that the rankings have a meaning beyond mere promotion.
Our rankings attempt to provide insight into the “education” at
different schools, and to reflect how that changes given different
types of student.
Q3. How is StudentsReview different?
built for individuals. Applying to college is a confusing
and poorly informed process, and rankings only tell you what
most people think, but do not provide insight about what
you might think. Individualized comments reveal the reasons that
someone might like or dislike their school, helping you to
learn if those reasons or opinions are relevant to you.
Q4. What about the “UnOfficial”, special purpose rankings? The unofficial
rankings are those that are either not generated from student
surveys, or generated with insufficient data to make reliable conclusions.
The purpose of including them is help provide an
informative list for students interested specifically in those schools.
They provide a “starting point” for those interested in schools
that primarily women's colleges, christian colleges, or solely on reputation
Q5. Why are the dynamic and official rankings different?
Dynamic Rankings are intended to provide a starting point for
students to find schools that a customized on 1 or
2 factors for them personally. Many users use it,
and the dynamic rankings need to be done in real-time.
For that reason, they are done with
as little computation as possible — meaning that very important
calculations such as distribution matching and normalization are left out
— to provide a fast approximation.
The Official rankings
however are a large cross-sectional analyses that perform normalizations, free
variable binding, apple-apple distribution matching, and overall require an enormous
amount of computation across a number of variables — it
really is not feasible in a realtime ranking system like
the dynamic ranking.
Because there is such a complex
mathematical difference between the two, there are bound to be
some significant differences.
To read in depth about how
the OFFICIAL rankings are performed, please go to Rankings Explained.
Q6. Schools can't be compared!!! Its like comparing apples and
It is true that schools are inherently non-comparable. Averaging
student opinion and other simple means of comparison simply do
not work because schools and their students differ in so
many dimensions. It is however, possible to model
the relationships between ALL of the students, and normalize to
analyze all schools on an equal playing field.
Please go to “Apples to Apples” in Rankings Explained to
read more about how we accomplished this.
X is ranked lower than another school but has high
It has to do with a number of factors —
primarily the amount of data, the error, and the missing
data. With low numbers of surveys, the scores are
more easily affected and have a greater chance of
being incorrect — so we measure that error and place
all schools at the bottom of their possible error range.
Q8. What about “Historically Black” Colleges?
Unfortunately we simply
do not have enough data to make any conclusions about
their relative rankings yet.
Q9. Which Statistical Package do you
use to analyze your data?
We use a version of the
R (The R Project) statistical analysis package that we've
specially customized to integrate with our systems.
Q10. What about
There are three primary sources of error in the
rankings, and they are: insufficient surveys, accumulated normalized difference,
bias difference/not a representative sampling.
Q11. Are the surveys aged?
What if someone surveyed 5 years ago?
Yes, the surveys
are aged over time and contribute less to the scores
are they become older; but we have found that universities
(in general) do not change all that quickly, so a
5 year old survey is still reasonably valid (about 60%)
Q12. How long did it take to create the
It took us over a month to analyze the data,
with many overnight sessions. Analysis started in late November.
Much of the time was spend discovering trends within
the data, co-dependencies, and just constructing a sound model
from which to analyze. Some companies may merely take
the averages of surveys, or combine the surveys in a
blanket manner for convenience, which is a trap we did
not want to fall in. The data cannot be
combined in that manner, and any analysis like this will
create highly misleading results.
Q13. Studentsreview only seems to have
really positive or really negative opinion! how can that be
used to create rankings!?
Not true - the comments may seem
to be highly bipolar, but a large portion of the
“middle” surveys simply lack a comment, making it seem like
there are only positive and negative opinions. But even
if the opinions were strictly positive or negative, then our
process of distributional matching, would match the distributions of the
students at different schools, so the schools would become comparable,
regardless of whether the opinion itself is positive or negative.
Q14. Can I get more information?
StudentsReview maintains an open policy, because we believe that for
data or analyses to be useful, it must be trusted.
To read more, please go to Rankings Explained.