Mass import

Status: implementation and iterative enhancements

• Introduction
  • The problem
  • The solution
• Markdown format
  • Markdown format structure
    • Possibilities of solution structure
  • Skills association
  • Exos formats
    • Text exo
    • Bool exo
    • Multiple Choice Question
    • Table exo
    • Exo with reference solution
    • Exo without solution
  • Resume of general prefixes meanings and format
• JSON format
• Import interface
  • General experience
  • Use cases

TODO: mass import should be named "bulk import" ?

#Introduction

#The problem

Copy-pasting manually content in a form to import hundreds of existing exercises is boring and very slow. This problem is an onboarding pain for teachers or students responsible to import the existing exercises.

#The solution

Provide different ways to import content massively, for different technical levels. The more technical solution that can already be used, consists of scripting the import via the API, but this requires to understand the Delibay format of exercises, to know how to use APIs with scripts, to have exercises in a machine-readable format...

The less technical solution is the Markdown format described below.
The more technical one is can be used when migrating content from another platform without manually copy-pasting, would require a script that can pull the existing content, create a big JSON file.

These Markdown or JSON files could then be dropped into the Import interface for the final import.

Note: there is no possibility to create exercises graphically right now, only the API can be used, and only teachers can create exos...

#Markdown format

This format is currently in progress to be implemented in the import interface... As a reminder, exercise's instruction and explanation supports Markdown and the word 'exo' is a shortcut for 'exercise'.

#Markdown format structure

The idea is to define a set of prefixes and keywords to describe exos information. First, let's look at an complete example with random exos, then we will look at the meaning of each prefix.

A fictive example of the format in a file sorting_algos_exos.md:

## Exo: Complexity
Instruction: Give the complexity of this piece of code
'''cpp
int k = 0;
for (int i = 0; i < N; i++) {
	for (int j = 0; j < N; j++) {
		k++;
	}
}
'''

Solution: O(N^2)


## Exo: Complexity of sorting algorithms
### Subexo:
Instruction: The complexity of the bubble sort is `O(N·log(N))` ?
Solution: false

### Subexo:
Instruction: The complexity of the quick sort is **in average** `O(N·log(N))` ?
Solution: true


## Exo: Stability in algorithms
Instruction: How do make a stable sorting algorithm unstable ? Explain your solution with bubble sort, include an example of code and explain why it would break stability...
Solution: [reference] 
We can write a lambda or function as the comparator function given to our sorting function:
~~~cpp
'''cpp
bubbleSort(v.begin(), v.end(), [](int a, int b) -> bool { return a <= b; });
'''
	//...
}
~~~
The stability of elements is possible when the 2 equal elements are kept in the original order. To change this behavior we need change the comparison sign from `<` to `<=` so it will swap values even when they are equal.

## Exo: Guess the sorting algorithm
Instruction: The following list `[4, 2, 7, 2, 9, 5, 4, 3, 3]` has been sorted with different 
sorting algorithms and stopped before the end of the sort.  
Guess which algorithm was used in each case.

### Subexo:
Instruction: **`[2, 2, 4, 5, 5, 7, 5, 7, 9]`**
Options:
- Bubble sort
- Insertion sort
- Merge sort
- Quick sort

Solution: Bubble sort

### Subexo:
Instruction: **`[2, 2, 3, 3, 4, 5, 4, 3, 9]`**
Options:
- Bubble sort
- Insertion sort
- Merge sort
- Quick sort

Solution: Insertion sort
Explanation: you can see elements that have been swapped between the start and the end of elements.

### Subexo:
Instruction: **`[3, 2, 3, 4, 9, 5, 4, 7, 3]`**
Options:
- Bubble sort
- Insertion sort
- Merge sort
- Quick sort

Solution: [multiple]
- Quick sort
- Merge sort

Note: code snippets have the standard Markdown notation ```, I just can't use it in the above example...

#Possibilities of solution structure

They are almost always 2 ways to define solution values (except for bool exo):
One solution...

Solution: hello

... vs multiple solutions:

Solution: 
- hello
- hey

#Skills association

To designate the skills associated to a part of exos inside a transcript file, there is a dedicated prefix # Skill:. Example:

# Skill: Introduction

## Exo: Exo 1
### Subexo:
Solution: true
### Subexo:
Solution: true
### Subexo:
Solution: false

# Skill: Testing
## Exo: Exo 2
Solution: true
## Exo: Exo 3
Solution: false

## Exo: Exo 4
### Subexo:
Solution: true
### Subexo:
Solution: true
### Subexo:
Solution: false

Then, exo 1 will be assigned to skill Introduction, and exos 2, 3, 4 will be assigned to skill Testing. These skills obviously have to exist to be referenced.

The prefix MUST be positioned before an exo, any other position will throw an error. Example of 2 invalid positions:

<!-- Error 1 -->
## Exo: Exo 1
# Skill: Introduction <!-- Error 2 -->
Solution: true

## Exo: Exo 2
### Subexo:
Solution: true
# Skill: Testing <!-- Error 2 -->
### Subexo:
Solution: true
### Subexo:
Solution: false

All exos must have an associated prefix, thus the first prefix MUST be at the very beginning.

<!-- Error: Exo 1 has no associated prefix -->
## Exo: Exo 1
Solution: true

# Skill: Introduction
## Exo: Exo 4
### Subexo:
Solution: true

Furthermore, all transcripts files MUST contain at least a # Skill: prefix at the beginning, because this is easier to define in advance what is the skill, instead of choosing it on the import interface. TODO: fix examples and tests for this new rule.

#Exos formats

#Text exo

The default exo type is text, the other types are not explicitly specified but are deduced from the context.

## Exo: Who is the creator of the Linux kernel ?
Solution: Linus Torvalds

#Bool exo

If the solution is true or false, this automatically makes it a boolean exo. There is no need to define the list of options this is always true or false.

## Exo: C is an interpreted language ?
Solution: false

#Multiple Choice Question

Just give the text of the correct option. If there is more than once solution, write all possibilities in a list.

First example (only one option can be checked and only one option is correct):

## Exo: Pets
Instruction: Select the only pet among these animals
Options: 
- cow
- cat
- chimp

Solution: cat

Second example (they are multiple correct option but only one to check):

## Exo: Pick one even number

Options:
- 2
- 3
- 4
- 5

Solution:
- 2
- 4

The solution must contain one or more options given under Options:

Multiple mode:

When correct options given in the list need all to be checked to consider the answer correct, the multiple mode is ON. To indicate this, you can add the keyword [multiple] right after the prefix:

## Exo: Which statement is correct ?

Options:
- C is an interpreted language
- C is a compiled language
- C is used in the Linux kernel
- C is mostly used for web applications

Solution: [multiple]
- C is a compiled language
- C is used in the Linux kernel

You can several checkable options but actually only one is correct, in this case instead you can directly give the option without a list (both notation will work):

## Exo: Pets
Instruction: Select **all** pets among these animals
Options: 
- dog
- cow
- chimp

Solution: [multiple] dog

But you CANNOT mix the 2 notations like this:

## Exo: Pets
Instruction: Select **all** pets among these animals
Options: 
- dog
- cow
- cat
- chimp

Solution: [multiple] dog
- cat

#Table exo

That's a more complex exo type, and it should be used only when there is no other possible formats, and line of the table cannot be divided in subexos with only a 1 or 2 values to give. See more on Best practices. Normal tables in Markdown can be used too inside the instruction, but they can't be filled. This is why there is a table exo format.

First a minimalist example
With a truth table for the OR logic gate, the third column (in bold) must be given by the student:

We would write this:

## Exo: The OR gate
Instruction: Fill the truth table of the OR logic gate

Table: [height:5] [width:2]
Header:
- A
- B
- A + B
Row:
- 0
- 0
- [text] 0
Row:
- 0
- 1
- [text] 1
Row:
- 1
- 0
- [text] 1
Row:
- 1
- 1
- [text] 1

Delibay will render text inputs on the last column. Header rows will be displayed in bold.

Another complex table with more columns:

Fill the blank cells with calculations about these different memory zones:

The student have to fill it like this:

We would write the following:

## Exo: Memory zones
Instruction: Fill the blank cells with calculations about these different memory zones:

Table: [height:4] [width:3]
Header:
- Start address
- End address
- Memory size in B
- Memory size in bits
Row:
- 0x900 
- 0xA4F
- [text] 0x150
- [text] 24kb
Row:
- [text] 0x15000
- 0x24FFF
- [text] 0x10000
- 100kb

Another complex table with empty cells and cells ranges:

Enter values in this memory after the previous assembly code:

The student have to fill it like this:

14/15 means here that 14 and 15 are 2 correct options. (Same logic for 9/09)

We can write the following text:

## Exo: Memory content
Instruction: Enter values in this memory after the previous assembly code:

Table: [height:5] [width:10]
Header:
- [none]
- [range:1...9]
Row:
- [empty]
- [text] 23
- [text] 92
- [text]
	- A4
	- a4
- [text] 12
Row:
Row:
- [empty:5]
- [text]
	- 14
	- 15
Row:
- [empty:5]
- [text]
	- 09
	- 9

Prefixes and keywords

• Table:: kinda "parent" prefix, all the following prefixes need Table: to be valid.
• [height:x]: total height of the table as x. Placed after Table: prefix.
• [width:y]: total width of the table as y. Placed after Table: prefix.
• Header:: header row of a table. It's possible to have multiple header rows.
• Row:: data row of a table
• [text]: there will be a text input and the solution is given right after. For multiple solutions, just give a list of values.
• [empty]: in a table to indicate text input that must be kept empty
• [empty:n]: n times [empty] cells.
• [none]: in a table to indicate an empty cell (without a text input)
• [none:n]: n times [none] cells.
• [range:x...y]: a set of cells with values shown from x to y. Really useful for the header with the number of each row.

Important: all undefined cells in table body are considered like [empty], and for table header like [none] cells. The height and width is only useful to help with big tables with a lot of cells to leave empty.

TODO: think about these ideas:

• define a Column: prefix for ranges of value like Column: [range:0...10]. Could be very useful is list of values in the table are more organized vertically... should ideally be possible to mix column and row. we'll see.
• remove the Instruction: prefix completly!
• rebuild and rethink the import interface

#Exo with reference solution

If the exo has no objective correctable solution, but a reference solution (one of the possible solution) that cannot be checked word by word on the student's answer, we use the keyword [reference]:

Instruction: Develop an efficient algorithm to insert an element an ordered set
Solution: [reference]
~~~cpp
//First let's define
vector<int> list;
// ... piece of code
~~~
This recursive approach is actually better because it reduces the complexity from O(N^2) to O(log(N)).

In this case, it's up to the student to give decide it's solution was correct or not. Or in the future, teachers would be able to give feedback on all students answers, or make a live feedback in class.

#Exo without solution

TODO

Things to know

• The end of multiline prefixes is defined at the next prefix position or at the end of file.
• Invalid format will be detected during the import, in case you forget them
• These prefixes and precisions is actually in a draft mode, they might change in the future
• Exo instruction is always displayed above instruction of child exo. Child exos are displayed (currently) one after the other, so don't quote another subexo because the student will not see it at the same moment.
• The order of exos given in the file will be the same after import, they may eventually be reordered in the Import interface.
• Prefixes are only recognized when placed at the start of a line and in the correct order.
• All values are trimmed (spaces, enter or tabs, are removed at the start and end)

#Resume of general prefixes meanings and format

Keywords

• [multiple]: in MCQ solution to say "multiple options can be checked"
• [reference]: in text exo's solution to say "this is only a reference solution, not one to check against"
• [single]: right after ## Exo: and used to define that the subexos must be displayed in a single page, not one after the other, because subexos are dependant and need to be done in the order.

Note: The idea of Markdown titles for exos and subexos is mostly to help visually with IDE Markdown highlighting to separate visually exos in long files. TODO: this is maybe becoming obsolete with the language support in VSCode with a dedicated extension ??

#JSON format

This is not implemented yet. To document later. Already shortly described in db.md...

#Import interface

#General experience

Now we have different ways to express the content of exos, let's discuss the way we can finally import them. As we are dealing with dozens or hundreds of elements, it's really important to build a nice experience to easily manage:

• Transcripts errors
• Server validation errors
• Network failures
• Import stopped by tab refresh or close
• Grouping exos in various skills
• Checking exos are in the correct skill
• Checking exos previews and try them before import

#Use cases

This complex interface can be used by teachers in various situations

1. The import of exos for the whole course: with a list of skills and a list of exos assigned to these skills. The skills associated with each exos or exos group, are mostly defined directly in transcripts files with the Skill: prefix.
2. The import of exos for a given skill: One or more transcripts files without indicating the destination skill. We need a dropdown with a list of skills to manually select it during the import process.
3. The import or update of an entire course: this is an advanced feature, to enable import and update from a remote git repos, or a given folder on disk (in these cases, the courses are maintained out of Delibay). It would require more validation and a file like course.md to define course information. It would require some kind of identifiers in transcripts to permit the update of edited exos only and not create duplicated exos.

TODO: do simple models on Excalidraw

Along the exo creation form, another page would exist take the Markdown files and create exos in mass.

Just imagine you have several Markdown or JSON files in a folder:

• you copy-paste the content or drag-and-drop them in the interface
• you can see a preview of all exos generated
• then you can apply last fixes in case of errors in the format
• secondly you can choose which skills are going to receive each exo or group of exos
• then you click on "Start import" and the import starts...

Everything is cached, so you can't lose exos in case the server fails, you refresh the page, or the import crashes at the middle. You can always restart with failed exos and fix eventual invalid content if the server has different validation rules. If needed, you can pause the import and come back later.