Add tooling for concept exercises

[ellnix]

Currently I have a couple things in mind:

• a generate command for creating concept exercises
• make the update command work with concept exercises (currently generate_exercise_files looks inside of exercises/practice)
• any helpful lints or checks I think of that configlet doesn't do (nothing specific so far)

Thought I'd open this issue first instead of just submitting a cold PR.

[senekor]

I think there are no "problem-specifications" for concept exercises, so an update command probably doesn't make sense.

[ellnix]

That's a really good point, they just happen to mostly be the same exercises as in problem-specifications, but we will have to maintain the tests ourselves.

[senekor]

Yeah. If there is a similar exercise in problem-specifications, the test cases there may serve as inspiration. But concept exercises are split in tasks and the tests need to map to these tasks. So it's not the same thing.

Btw I saw there's a a configlet command to add a concept exercise, not sure how much of the heavy lifting that already does.

[senekor]

But concept exercises are split in tasks and the tests need to map to these tasks.

The test runner doesn't support this yet, as there is no syllabus. Ideally, the information which task a test belongs to is embedded in an easily-parsable way. For example in the function name:

#[test]
fn task_1_foo() {}

#[test]
fn task_2_bar() {}

I'm open to alternatives.

Once a decision is made, I'll add the functionality to the test runner. For reference, this is the relevant issue.

Whatever we land on, we should have a test to make sure this rule is followed for all concept exercises. The test should include a comment that the test runner relies on this format.

[ellnix]

The test runner doesn't support this yet, as there is no syllabus.

I was going to ask about this actually, since Exercism's website strangely says the test runner supports the latest version of the spec, maybe that's something different:

I'm open to alternatives.

Perhaps different modules? It might make sense to have tasks even be on different files sometimes.

mod task_1 {
    #[test]
    fn foo() {}
}

mod task_2 {
    #[test]
    fn bar() {}
}

I imagine that the test output would be easier to parse, since you'd have task_\d+::test_name instead of task_\d+_test_name.

Whatever we land on, we should have a test to make sure this rule is followed for all concept exercises. The test should include a comment that the test runner relies on this format.

Fair enough, noted.

In addition I would like to propose that no test title should have the string /task_\d/ appear more than once. Ideally the word task should not appear on them at all:

// annoying: running `cargo test task_1` runs a task_2 test
#[test]
fn task_2_todo_list_add_task_1 {}

// less annoying, but would be annoying if you were trying to navigate tasks through text search
// for instance running `/task` in vim and going through the matches
#[test]
fn task_2_give_bob_a_task_and_hear_him_complain {}

I'm mainly opposed to the first example with task_1.

[senekor]

I was going to ask about this actually, since Exercism's website strangely says the test runner supports the latest version of the spec

The test runner reports to be at version 2, so I'm assuming the website says it's on the latest version specifically because there is no syllabus.

Perhaps different modules?

That seems nice, but I think it would be a little harder to parse. Because the test runner associates a test with its source code (so users can be displayed the source of the test that failed), I'm actually parsing the test file itself. This is currently done in a very simple fashion, just greedily looking for patterns like #[test] in the text.

On the other hand, it might also be easy to look for mod \w+ { and remember the current module.

I would like to propose that no test title should have the string /task_\d/ appear more than once

Fine by me, feel free to add a test for that when the time comes.

[ellnix]

That seems nice, but I think it would be a little harder to parse. Because the test runner associates a test with its source code (so users can be displayed the source of the test that failed), I'm actually parsing the test file itself. This is currently done in a very simple fashion, just greedily looking for patterns like #[test] in the text.

If that's the case then please by all means proceed without modules. We should have the implementation be as simple as possible.

Fine by me, feel free to add a test for that when the time comes.

👍