Skip to main content
We are currently relaunching the website.
In the interim, use below link to access some of the prior work:
https://www.crcv.ucf.edu/REU/paper2.php

Mentoring Undergraduates In Computer Vision Research

 

Mubarak A. Shah and Kevin W. Bowyer


 

Abstract

The future of our society will be shaped by the young and talented minds going through our colleges and universities today. During the last ten years we have worked with over seventy undergraduate students from a half dozen institutions in Florida under Research Experiences for Undergraduates (REU) program funded by NSF. A large fraction of our REU participants have been able to prepare a paper for submission to a conference, have the paper accepted and then attend the conference to present the paper. Several participants have even accomplished substantial enough research to also result in journal publications. Many of our past participants are now pursuing graduate studies at various institutions. In this paper, we describe our REU model in detail, discuss some examples of student success, and summarize our observations.

 

Introduction

The NSF predicts a shortfall of around 700,000 scientists and engineers by 2010. In 1977, there were 4,000,000 high school sophomores in the U. S., 730,000 of these students expressed interest in science and engineering careers. In 1980, when this cohort entered college, 340,000 retained their interest. By 1984, 206,000 had actually graduated in scientific or engineering disciplines. Only 61,000 of these men and women entered graduate school in science or engineering. By 1992, just 9,700 will graduate with Ph.D.s. To have only 0.2% of these students end up with doctoral degrees does not seem promising! On the other hand, according to the report by the U.S. department of labor, between 1986 to 2000 the computer and data processing industry is projected to be the fastest (76%) growing of all industries in the economy. The Chronicle of Higher Education for February 26, 1997 states that: “The United States may lose its lead in the information- technology industry because it isn’t producing enough educated people to work in the fast-growing field, a report released recently warns.”

In 1987, NSF started a new program, Research Experience for Undergraduates (REU). The goal of the REU program is to encourage talented students to pursue graduate studies and realize their full potential in this regard. We have received six NSF REU site awards totaling over $800,000 during the last ten years. The first grant was in 1987, with the first author as the sole P.I.. Since then we have received grants in: 1989 (P.I. Mubarak Shah, Kevin Bowyer, G. Krishnan), 1990 (P.I. Mubarak Shah, Kevin Bowyer, G. Krishnan), 1991 (P.I. Mubarak Shah, Kevin Bowyer, G. Krishnan ), 1992 ( P.I. Mubarak Shah and Kevin Bowyer) funded for two years, and 1995 (P.I Mubarak Shah, Kevin Bowyer, Louise Stark and Niels Lobo) funded for three years. Approximately one hundred undergraduate students from a half dozen institutions in Florida (e.g., Rollins College, Stetson University, Eckerd College, UCF, USF, FIT) have participated in this program. This paper describes our experience with NSF-funded REU projects over the last ten years.

 

General Philosophy

We believe that the best way to achieve the goal of the REU program is to involve the students as a part of a successful, active research group. This exposes them to the intellectual excitement that is involved in research, encourages them to think creatively and independently, and helps them to develop the skills necessary to work on research projects. In addition, students are exposed to professional meetings, learn to assimilate the latest research from reading and discussing recent papers, and learn how to write up and present the results of their own research.

It is obvious that an undergraduate research experience that lasts only a few months does not allow sufficient time to complete a serious project. Hence, such a short project would not allow participants the chance to achieve the feeling that they have made a significant original contribution. Since this feeling of accomplishment is the primary motivation of most researchers, we believe that the REU experience must be structured so that each student has the maximum opportunity to successfully complete their chosen project. Hence, our REU experience is spread out over (at least) one calendar year, typically as a summer/fall/spring sequence.

From our experience, we have found that interaction among students from different institutions greatly increases the diversity of ideas and the quality of the experience. During the academic year, however, guidance and supervision is difficult if the student and the faculty member are at different institutions. Consequently, we feel that it is important to have a faculty member at each institution directly involved with, and responsible for, each student. The faculty member can then easily supervise the student’s academic year course load and be available for advice and guidance. We feel that our management strategy for the program has been demonstrated over the past ten years to be quite successful.

 

Selection of Participants

We primarily look for students who are juniors or seniors in the Fall semester of each year, who have a GPA of at least 3.25, and who have a strong background in Mathematics and basic Computer Science. After an initial screening based on written applications, the faculty members meet individually with the potential participants and decide which should receive offers.

While we have found that “word of mouth” from current and past participants is the most effective recruiting tool, the following efforts are also made:

 

  • advertisement of the opportunity in the campus newspapers,
  • notices posted at various places on the campuses,
  • announcements made in the junior level classes,
  • announcements distributed in Tau Beta Pi (Engineering honor society), UPE (Computer Science honor society), IEEE and ACM student chapters,
  • juniors majoring with a GPA of 3.25 are sent a letter inviting them to apply,
  • students in the {\em Honors} programs are contacted, and encouraged to apply,
  • current REU participants are asked to give a talk about their experience at SWE (Society of Women Engineers) and SME (Society of Minority Engineers) meetings in the spring semester, Faculty/Student Seminars, and
  • we personally contact women, minority and especially-talented students and encourage them to apply.

On the average, every year we receive about twice as many applications as the number of students we can select to participate. However, that number has recently been decreasing because good employment opportunities exist for computer science majors. The students who are not selected to participate typically either (1) do not have enough background in Mathematics and programming, or (2) are unable for academic and/or financial reasons to devote subsantial time in the summer to the REU program.

 

Student Activities

One of the important goals of this experience is sparking the student’s interest in scientific research as early as possible through student-faculty and student-student interaction. At the same time, we feel that each student must have sufficient time to mull over problems and somewhat independently achieve a reasonable level of solution of a problem of their own choosing. Students are able to devote a major portion of their time to the project during the summer, since they are not involved in regular courses. During the academic year, continued student involvement is ensured by the faculty member at each institution. The professor(s) at each institution supervise the students through their registration in elective Computer Vision, independent study and senior project courses and through weekly individual and research group meetings.

Our last proposal to NSF for REU covers a period of three years. Our experience has shown that there is a need for flexibility to deviate from the “standard” one calendar year in certain situations. Sometimes an excellent prospective participant has an academic or co-op schedule which does not allow a continuous 12-month participation. In such cases, it would be useful if the “REU year” could be spread over a longer calendar period. Sometimes, also, one of our participants does excellent work on his/her topic and does not graduate for as much as another year after the end of the 12-month REU schedule. In these cases, it is useful to offer the student a chance to stretch their “REU year” to some longer period. Lastly, a participant may be forced, due to personal reasons, to drop out of the program, and then a new participant must be found to take their place. We regularly see perhaps three special cases out of ten participants.

 

Student-Faculty Interaction

The appropriate level of student-faculty interaction and supervision of the participants is achieved by the following means:

  • Weekly Individual Meetings: Each faculty holds a weekly individual meeting with each of the REU participants to discuss their research project.
  • Weekly Research Group Meetings: Each faculty holds a weekly seminar-style meeting with his research group, in which current research papers are discussed, upcoming presentations by group members are rehearsed and discussed, papers by group members are “pre-reviewed,” participants give presentations on the status of their project, etc.
  • REU Group Meetings: The whole REU group, consisting of faculty and REU students meet once a semester. This meeting lasts for one day and is scheduled on weekends. The meeting is structured into three sessions consisting of presentation, demonstration, and discussion sessions. During the first session the student present their projects. These presentations include a brief introduction of the problem, results to date, and future plans. Next, the students are able to demonstrate their projects in the Computer Vision lab. Finally, we have an open discussion session where each student is encouraged to comment on other projects.
  • Electronic Mail: Students are able to communicate with fellow students, the graduate research assistant and faculty at their campuses as well as at other campuses using electronic mail.

 

Journal Club

We believe that REU participants should also be exposed to other research problems in Computer Vision, beyond their own area of concentration. In order to achieve this, we organize a “journal club”, a paper discussion session during the spring semester. By the start of the spring semester, students have a fairly good idea of basic concepts in computer vision. Each time a research paper is selected and assigned to a student. That student study the paper carefully and present it in the journal club. The other students also read the paper and prepare questions and comments for discussion with the student responsible for that meeting. That way students will follow the content in the paper, and will also be trained to be critical in a constructive sense. These sessions are moderated by the faculty at their respective universities. This is an important and effective means of getting students used to making presentations about technical material and of teaching them how to read and comprehend journal papers.

REU Speaker Series

The aim of this series is to allow the REU participants to meet and interact with some well-known researchers in the field. Typically, during the morning the speaker gives a talk on some aspect of his or her research. We arrange a lunch or dinner for the speaker to which the whole group of REU participants are invited. Finally, in the afternoon we hold an informal discussion session where the REU participants are free to ask questions related to any topic in computer vision. We attempt to schedule at least two guest speakers every year.

Participation in Professional Meetings

Attendance at professional meetings gives the REU participants exposure to the well-known researchers in the field, provides an opportunity to see good and not-so-good research presentations, and provides an opportunity to assimilate the latest research results. Therefore, we take REU participants to at least one professional meeting related to computer vision. We are fortunate in that our geographic location make this possible at a reasonable cost. The annual SPIE Intelligent Information Systems meeting held in Orlando each spring has at least one conference devoted to robotics and computer vision. This is our default option. In many years, a more focused meeting is held close by. For instance, in 1994 and 1996 the IEEE Workshop on Applications of Computer Vision was held in Sarasota, and in 1995 the IEEE Workshop on Computer Vision was held in Miami. Sometimes the students are also taken to a conference outside Florida, for example REU students were taken to ICCV9 ’95 in Boston. REU students who get a paper accepted in a conference are often sent to the conference with travel funds from the grant.

Student Preparation and Follow-Through

Students receive background preparation for their research project through the computer vision short course in the summer. All students devote a substantial portion of their time during the academic year to the research project, since they have an independent study, senior project or senior thesis supervised by one of the faculty. We help the students as they apply for graduate school admission during the academic year. We encourage them to take the GREs early enough that they can take them again if they do not do well initially. (It has been our experience that students taking the exam a second time often improve their score by over a hundred points.) We advise them about which schools have good programs in what subareas. We also encourage qualified students to apply for graduate fellowships awarded by NSF, NASA, ONR, AFOSR, the Florida Endowment Fund program, and the State of Florida. We facilitate interaction with outside researchers visiting any of the campuses during the academic year, so that students develop the contacts necessary for admission to the best schools. We also contact company representatives and have them arrange interviews and site visits for students interested in industry.

Yearly Schedule

June 1 — July 15
The first major activity for the students is the short course in Computer Vision. Since, we assume that the students do not have prior background in Computer Vision, this course quickly introduces them the subject. The topics covered include: imaging geometry, edge detection, region segmentation, 2-D shape, stereo and shape from shading, motion, etc. The lecture component of this course is a one-day meeting each week for six weeks. These lectures are shared by Professors Shah and Bowyer. During the remainder of each week, the students re assigned some readings in the area covered by the lecture material and be given one or two short programming projects which support the material. In the first two weeks, some extra instruction in C programming, X windows and use of the Unix workstation are given. This method of scheduling and presenting the short course was arrived at after review of the 1989/90 project and discussions with the participants about the effectiveness of the six straight days format versus the one day a week for six weeks format.

July 15 — August 15
In the third week of July, each of the faculty members suggest a number of possible projects and research topics. Each student is able, within some limits, to {\bf choose} a project topic rather than have one {\bf assigned}\/. We feel that this is important in terms of having the student feel that the project is “theirs.” Also, we are careful to structure projects so that they have several possible levels of completion. This ensures that each student will be able to complete some (possibly simplified) version of their project and experience the resulting feeling of accomplishment. It also allows the more talented or ambitious student who works hard enough to achieve a substantial research publication. This approach has proven successful with our past REU participants. After each student has a specific project, student is then given background reading assignments and program projects for their chosen project. By the end of this period, they should be prepared to write a detailed prospectus of their proposed research project.

August 15– August 30
In this time period, we hold joint meetings at which each REU participant present their proposed research project plan. The other members of the group provide comments and constructive criticism. Then the participants are able to revise their plan and pursue their projects full-time until the beginning of the fall semester.

Academic Year
During the Fall and Spring semesters, the students continue working on their individual projects. There is one meeting of the entire REU group during each semester, to review the projects and discuss the problems encountered and the progress made. The students make presentations of their projects, and when appropriate demonstrate their results on workstations. Further, the participants be taken to at least one conference at which Computer Vision is a major topic. At the end of the year, each student is asked to write a comprehensive report describing the project. Students with publishable results are encouraged to write conference and/or journal papers.

Student Success

Approximately one hundred undergraduate students from a half dozen institutions in Florida (e.g., Rollins College, Stetson University, Eckerd College, UCF, USF, FIT) have participated in this program. Many of these students have co-authored papers, and most of them have been admitted to graduate programs. Here are some examples.

  • A 1994/95 REU participant, Shawn Dettmer, worked on Lipreading project. His paper was accepted in {\em International Workshop on Automatic Face and Gesture Recognition}, Zurich, 1995. He was admitted to the graduate School at UCF. He received a DSR enhancement fellowship for two years. He completed his M.S. in spring of 1997. Currently, he is a Ph.D. student.
  • A 1993/94 REU participant, Jim Davis, worked on gesture recognition for his REU project. A short version of the paper about Jim’s work was accepted in 1994 European Conference on Computer Vision (acceptance rate 18%), and a long version appeared in a journal. He completed his Honors in The Major thesis in 1994. Jim was admitted to the graduate program at MIT with full fellowship. He completed his M.S. in 1996. Currently, he is a Ph.D. student at MIT Media Lab.
  • A 1993/94 REU participant from USF, Chris Doss, worked on “Comparative evaluation of pattern recognition techniques for detection of microcalcifications in mammography”. His paper appeared in International Journal of Pattern Recognition and Artificial Intelligence. Christopher is now in the graduate program at North Carolina State University.
  • Another 1993/94 student Warren Macchi worked on “Interreflections with rough surfaces”, and his paper appeared in 1994 International Conference on Pattern Recognition. Warren received a three year fellowship from NSF for his graduate degree at UCF. He passed his Ph.D. qualifying exam in the Spring of 1997.
  • A 1992/93 REU participant from Eckerd College, Chris Stimac, worked on “Refinements of DCN neural network algorithm.” Chris entered graduate school in Mathematics at UC Berkeley.
  • A 1991/92 REU student from USF, Melanie Sutton, worked on “Reasoning about function to achieve generic recognition of rigid 3-D shapes.” This led to graduate work which was eventually published in the journal Pattern Recognition, and received honorable mention for Pattern Recognition’s best paper of 1994 award. Melanie is now on the faculty of Computer Science and the University of West Florida.
  • A 1989/90 REU participant, Matt Lavoie, who received the Outstanding Research award for his Honors in The Major thesis (REU project), was invited to make a presentation on his REU project at NSF in March 1990. Matt published one journal and one conference paper. He successfully completed his MS at UNC, and now co-owns a company.
  • A 1989/90 REU participant, a USF student, Maha Sallam, had a paper describing her REU work accepted by the Pattern Recognition Letters journal and then had a paper describing her continued work accepted as a “long paper” at the 1990 International Conference on Computer Vision (ICCV) (acceptance rate 5%). She completed her Ph.D. at USF and is working on a medical imaging start-up company in the Tampa Bay area.

 

Observations

Here are some observation from ten years of experience with REU.

  • The department and University strongly encourage this activity. Each year we receive some matching funds from the University for this grant in terms of course release during the academic year, graduate student support, travel support etc. The University newspaper has published at least four stories about REU, twice on the front page. In 1991, we had had a luncheon for honoring REU students with president and dean in attendance. REU is considered positively for the tenure and promotion process, and some other awards (e.g., Teaching Incentive Award (TIP)). REU stands out from other standard things faculty do: teaching, research and service.
  • From the faculty point of view, REU is very time consuming. In particular, summer is very hectic. We are tied up to stay almost whole summer at our campus, because it is crucial for REU students. The problem is that the REU grant does not pay for any part of summer salary; we have to get support from other grants, or teach during the summer. The direct benefit of this is that on the average every year at least one student produces a good paper. We also get potential graduate students.
  • Some students are either discontinued or drop out from the program. Some of those students find out that they are not really interested in research, and others find that financial concerns are playin a big role in their life and that they can make more money at a part-time job.
  • We have realized that there are two main deficiencies in the students’ background relative to preparation for research: Mathematics and Programming.
  • We feel summer is the crucial time, because during fall and spring students get very busy in their course work. If a student does well during the summer, and have defined a good project, and started to get some results, then he or she will do well during fall and spring.
  • We have observed that some bright students are only interested in superficial things. They will write some code to experiment something. But, when things get serious, and become mathematcial they back off.
  • Some students do not do that well during REU, but later they realize their potential, and become heavily interested in research, and go to graduate school.

 

List of Journal Papers and Book Chapters by REU Students

The following is a list of publications co-authored by past participants in the REU program. The names of undergraduates are shown in bold face.

  • James Cryer, Ping-Sing Tsai and Mubarak Shah, Combining Stereo and Shading, Pattern Recognition, Volume 28, No. 7, pp 1033-1043, July 1995.
  • Li Nan, Shawn Dettmer, and Mubarak Shah, Visually Recognizing Speech Using Eigensequences, chapter in Motion-Based Recognition Kluwer Academic Publishers, 1997.
  • Woods, K.S., Doss, C.C, Bowyer, K.W., Solka, J.L., Priebe, C.E. and Kegelmeyer, W.P., Comparative evaluation of pattern recognition techniques for detection of microcalcifications in mammography,International Journal of Pattern Recognition and Artificial Intelligence, pp 1417-1436, December 1993.
  • J. Davis and M. Shah, Visual Gesture Recognition, IEE Vision, Image and Signal Processing, 141(2):101–106, April 1994.
  • P. Tsai, M. Shah, K. Keiter, and K. Kasparis., Cyclic Motion Detection. Pattern Recognition , Vol. 27, No. 12, 1994.
  • Jay Hackett and Mubarak Shah, “Multisensor Fusion: A perspective”, chapter in Trends in Optical Engineering, 1993.
  • J. Hackett, M. Lavoie, and M. Shah. Object recognition using multiple sensors. Journal of Information Sciences and Technology, 1992.
  • Krishnan Rangarajan, Bill Allen, and Mubarak Shah, “Matching Motion Trajectories”, Pattern Recognition, Vol. 26, No. 4, pp 595-610, April 1993.
  • J. Hackett, M. Lavoie, and M. Shah, “Three Dimensional Object recognition using multiple sensors”, Journal of Information Science and Technology , October 1992.
  • J. Hackett, and M. Shah. Segmentation using range and intensity data. Optical Engineering, pages 667–674, June 1989.
  • K. Gould, K. Rangarajan, and M. Shah, “Detection and representation of events in motion trajectories”, chapter in Advances in Image Processing and Analysis, (editors: Gonzalez and Mahdavieh), Optical Engineering Press, June 1992.
  • M.Y. Sallam, and K.W. Bowyer. Generalizing the Aspect Graph Concept to Include Articulated Assemblies, Pattern Recognition Letters, pp 171-176, March 1991.
  • Sutton, Melanie, Stark, L. and Bowyer, K. Reasoning about function to achieve generic recognition of rigid 3-D shapes, accepted to appear in Pattern Recognition.
  • Bowyer, K.W., Sallam, Maha, Eggert, D. and Stewman, J.H. Computing the generalized aspect graph for objects with moving parts, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 605-610, (June 1993).
  • Woods, K.S., Doss, Chris, Bowyer, K.W., Solka, J.L., Priebe, C.E. and Kegelmeyer, W.P. Comparitive evaluation of pattern recognition techniques for detection of microcalcifications in mammography, International Journal of Pattern Recognition and Artificial Intelligence.Last modified: Wed Oct 1 09:31:51 EDT 1997