Test Types for CS Courses

In our courses we have three types of tests for class projects/assignments: public, release, and secret. Keep in mind some projects may only have a subset of them. Additional information about each kind of test is provided below.

• Public tests → You have access to these tests. That means you know what the test is about, and what are the expected results. This information is provided when the project is posted, and in some projects the actual code for the test is provided along with the results. The submit server executes these tests, and you should always verify you pass the tests in the submit server, even you passed them locally (e.g., in Eclipse or in the grace cluster).
  
  
• Release tests → Unlike public tests no information is provided about these tests. That means you do not have access to information that describes what the test is about, and what are the expected results. In some projects the instructor might provide a meaningful name for a test and that can help you in the process of figuring out what is wrong with your code, but often the names used are not descriptive.
  
  There are several goals for release tests. First, it allows you to verify the functionality of your project. Second, it encourages you to work early on your projects. Release tests results, unlike public tests results, can only be seen a limited number of times. For example, when a project is said to have three tokens (assuming a 24-hour period) it means that you have three opportunities to see (in the submit server )the results of release tests in a 24-hour period. Notice that after you use a token (i.e., after you do release testing) you need to wait 24 hours for that token to be used again. For example, suppose you submit your project and do release testing at 1 pm. After release testing you realized you are not passing one of the tests. You then modify your code, submit, and release test again at 2:15 pm. After release testing for the second time you still do not pass all tests. Once again, you modify your code and release test at 3:30 pm. At this point you will not be able to release test again until 1 pm (the first token you used would have regenerated by then).
  
  As you can see, the sooner you start working on your projects, the more opportunities you have to do release testing. Notice that the results of release tests are always computed and used to generate your project grade whether you release test your project or not. If for some reason you forgot to do release testing, we will run the tests anyways. Of course, you could have fixed problems had you release tested your project; you just lost the opportunity to do better in the project (in case something went wrong with release tests).
  
  Notice that in most projects you can only perform release testing after you pass all the public tests.
  
  
• Secret tests → These are tests you don't have any access to. You do not know what the tests are about and you will not be able to see their result. You will know whether you passed these tests once you receive the grade for your project.

After you received a grade for project, you are welcome to see a TA during office hours to find out additional information about release and secret tests you may have failed.

Web Accessibility