Goal

Tutoring Strategies: The goal is to understand how LLMs can facilitate interactive and personalized learning in Python programming. Students will learn to use LLMs to analyze code, interpret syntax and semantics, and address error messages through their interpretation and debugging suggestions. They will also design prompts to receive hints rather than direct answers, and leverage LLMs to create additional questions and tasks to reinforce learning of the key concepts.

The long-term goal is to learn how to program independently and use LLMs to enhance the efficiency. To use AI-generated help effectively, it’s important to first build your own programming skills. By practicing coding yourself, you’ll develop a solid foundation, improve your problem-solving skills, and become more self-reliant. Understanding the details of Python programming without overly depending on LLMs will prepare you to use LLMs more effectively as a helpful tool. Otherwise, you will be a copy-pasta pirate without any understanding of what you are doing.

Analogy: Imagine you’re trying to build muscle and get stronger. To do that, you can’t just watch someone else lift weights at the gym and expect to see results in your own body. The real progress happens when you put in the effort yourself and gradually increase the resistance as you grow stronger. The more you challenge yourself, the more muscle you’ll build over time. Now, think of learning programming in the same way…

Disclaimer: LLMs are not always reliable sources of information and may occasionally provide incorrect responses. However, for common and well-known topics covered in this workshop, the likelihood of mistakes is low. Always think critically about advice from LLMs and check the accuracy of the information when needed. If you want to learn, always remind an LLM to provide hints not answers or solutions.

Setup

Motivation

Before starting exercises, let us discus and create the setup of our learning framework.

Systematic learning

We recommend learning programming from a textbook or in a course. For this workshop, we assume that you are following this structured approach. While it’s possible to learn using only a model, you will need to actively track your progress.

You can start by informing the model about your current level and what you expect from using it as a learning tool.

I am a complete novice to programming and would like to learn Python with your assistance as my teacher and tutor. Could you recommend some essential topics that I should cover to acquire basic skills?

You can then allow the model to guide you, asking for further explanations or exercises to enhance your skills. As you progress, you can direct the model to address problems specific to your field of study.

Personal tutor

At first, it is useful to specify what role we want an LLM to play and what are our personal needs and expectations. In this session, we want it to be our individual tutor adjusted to our knowledge and skill level. Therefore, we may start with declaring this role and a brief calibration exchange.

Act as my personal tutor and help me understand the provided examples and concepts. Start by asking me questions that help you gauge my level of understanding. Ask me one question at a time and wait for a response before moving on. Once you have calibrated my current level of knowledge, we will start with the learning process. In this process provide me with suitable explanations, examples, and analogies about the content. Keep your responses tailored specifically to me and my understanding level. If I ask you a question, first help me to find an answer by asking leading questions and giving hints. When answering my questions, answer only one open-ended question at a time. Examine my understanding by asking me to explain my thinking and use my own words. Be encouraging but keep going until I have mastered the content.

Let us start with finding out what I already know about programming in Python.

Additionally, you may provide some information about yourself. This will give you not only more personal experience (e.g. name) but also responses tailored to your background (e.g. study field, skill level) and interests (e.g. hobby). However, be careful to not provide any sensitive data to an LLM or in general, elsewhere.

We will take a few minutes for the calibration process, but later you may continue the exchange as long as it feels useful. At the end of a each stage, ask an LLM to summarize the session.

Please create an initial prompt that I can use to provide concise guidelines for a large language model serving as my individual tutor in future conversations. Include the personal tutor role description that I provided at the begining of this conversation. Include the lerner profile, summarizing my current level of knowledge and mental capacities.

Keep the chat with the role description open. As we progress with the exercises, you may want to extend it with additional information about your skills. After duing it, just ask an LLM to generate a promt with role description.

Code Interpretation

Motivation

Just like in natural languages, reading code is generally easier than writing it. That’s why we’ll begin this workshop by reading existing source code. Additionally, a significant part of programming involves understanding and analyzing source code.

Code

As an example, we will use a program from the first session of 198801, which is a modified version of the example from the Python for Everbody textbook.

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]

counts = dict()

for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None

for word, count in list(counts.items()):
    if bigcount is None or count > bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

Role

Ask an LLM to play a role of your personal tutor adjusted to your skill level as desciribed at the begining of this tutorial. Just cpy the role prompt before you start woring on the actual task.

If an LLM starts to behave in an undesired way, you may go back to the calibration process to get a better role prompt. You may include it in the current chat or create a new chat to get a clean context.

Purpose

At first, we may ask for the purpose of the course code (using an adapted version of Code clarifier prompt).

Your task is to take the code snippet provided and explain it in simple, easy-to-understand language. Break down the code’s functionality, purpose, and key components. Use analogies, examples, and plain terms to make the explanation accessible to someone with my current coding knowledge. Avoid using technical jargon unless absolutely necessary, and provide clear explanations for any jargon used. The goal is to help the reader understand what the code does and how it works at a high level.

Please help me understand the given CODE. I will always refer to it as CODE. Here is CODE:

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]

counts = dict()

for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None

for word, count in list(counts.items()):
    if bigcount is None or count > bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

Please explain to me briefly, in one sentence, what is the purpose of CODE.

Semantics

Once we know what the program is doing we may look into the details.

Depending on your level, you may either ask an LLM for details (1) or try to figure out the meaning of particular parts by yourself (2).

  1. If you are a novice, for the first task, it would be easier to start with passive learning to get an explanation and next, examine your comprehension. For this purpose you may use the following prompts.

    Now, please give me more details about the semantics of particular parts of CODE.

    As a response, we may expect the code to be split into chunks together with their explanations. Next, we may ask for more details

    Please give me more details about component 3.

    After reading the response, we may request understanding questions:

    I think I understand this. Could you ask me a few questions to check if I got important points?

    Answer questions or ask for correct answers. Repeat the steps till you achieve the desired level of understanding.

  2. If you have programming skills, you may go for active learning starting with the following prompt.

    Now, I want to figure out myself what is the purpose and meaning of particular lines of the CODE. Please lead me through this process with questions and hints.

Terminology

When you start learning programming, you may find some technical terms unclear or confusing. Textbooks often include glossaries that explain these terms. Additionally, you can use a model to:

  1. provide an explanation,
  2. rephrase an explanation to suit your understanding,
  3. illustrate concepts with examples,
  4. clarify differences between similar terms.

Below you have prompt templates and examples that could be used for this task. You may try the provided example prompts, take terms from an example glossary or used in the chat by an LLM.

  1. Explanation: To understand a specific term T, you can use this prompt:

    Can you briefly explain T at my current programming level?

    For instance, you might ask for an explanation of ‘machine code’ this way:

    Can you briefly explain ‘machine code’ at my current programming level?

  2. Rephrasing: If the explanation E is unclear, you can ask the model to simplify it:

    I read that T is E but I find it confusing. Could you provide a brief and simple explanation to help clarify this for me?

    For example, if the the explanation of the ‘machine code’ as the lowest-level language for software, which is the language that is directly executed by the central processing unit (CPU) is too complex, you can ask for clarification:

    I read that ‘machine code’ is the lowest-level language for software, which is the language that is directly executed by the central processing unit (CPU) but I find it confusing. Could you provide a brief and simple explanation to help clarify this for me?

  3. Examples: To deepen your understanding, you can request examples:

    Give me an example of T

    For example, we may get an example of the ‘machine code’:

    Give me an example of ‘machine code’

  4. Differentiation: If you’re confused by similar terms, ask the model to explain the differences:

    I read about T1 and T2 . I want to be able to use the terms precisely, but I do not fully understand the difference between them. Can you explain it shortly in simple terms?

    For example, if you are learning ‘main memory’ and ‘secondary memory’ (glossary), this approach can help clarify the distinctions.

    I read about ‘main memory’ and ‘secondary memory’ . I want to be able to use the terms precisely, but I do not fully understand the difference between them. Can you explain it shortly in simple terms?

    Another terms you may try are the interactive and script mode, a syntax, logic and semantic errors, a parameter and an argument, a function and a method.

Comments

Before finishing the conversation, we may ask the model to provide us with a version of the code containing explanation comments, especially for parts that were unclear to us. We may store this version on our computer for the future reference.

Please add explanatory comments to CODE. In general, use brief explanations except for the more difficult parts for me to understand.

You may be more specific about what you want to have explained briefly and what more extensively. If you are not satisfied with the initial response, work iteratively to achieve the desired outcome.

Exercise

Repeat this process with your own program, one from the Internet or ask the model to generate one for you.

Please give me an example Python program for my current level. It should contain a function and an if-statement but no loops. Please do not add any explanation comments to it, as I want to use it to practice the interpretation of source code.

If the program is not corresponding the desired difficulty level, ask for something more difficult or easier, including or excluding particular concepts (e.g. conditional execution, loops).

Start from setting the context and follow the other steps of the example.


Error Interpretation

Motivation

At times, a program may fail to run due to syntax errors, which are mistakes in the structure of the code. Other issues may also prevent it from executing properly. When such errors occur, the computer generates a “traceback”. This is a report that details where the errors happened in the code. For beginners, interpreting a traceback can be challenging, but learning to recognize and understand common error messages is crucial. Developing this skill will significantly accelerate your ability to troubleshoot issues and enhance your overall programming efficiency.

Code

This is a modified code, where an error was introduced. You may pass it to an LLM together with the traceback message or not.

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]


for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None

for word, count in list(counts.items()):
    if bigcount is None or count > bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

Traceback

Let us assume that we used obtained the following traceback:

Traceback (most recent call last):
  File "/home/joanna/usr/space/tmp/demo.py", line 10, in <module>
    counts[word] = counts.get(word, 0) + 1
NameError: name 'counts' is not defined

Role

Ask an LLM to play a role of your personal tutor adjusted to your skill level as desciribed at the begining of this tutorial. Just cpy the role prompt before you start woring on the actual task.

If an LLM starts to behave in an undesired way, you may go back to the calibration process to get a better role prompt. You may include it in the current chat or create a new chat to get a clean context.

Context

As with the code, we will set the context for our conversation first. We may continue in the same chat session (if the error is related to the code) or start a new one (otherwise).

Please help me understand a traceback I got while executing a program. I need explanations at my current level. Please wait for my questions, in which I will refer to the given traceback as ERROR. During our conversation, please just tutor me without providing extensive explanations or solutions until I explicitly ask you for them.

Here is ERROR:

Traceback (most recent call last):
  File "/home/joanna/usr/space/tmp/demo.py", line 10, in <module>
    counts[word] = counts.get(word, 0) + 1
NameError: name 'counts' is not defined

Structure

First, we need to read how to interpret a traceback, so we can ask the model to support us in this process.

First, I would like to know how I should approach interpreting a traceback and what I need to pay attention to to understand what caused a problem while executing a program. Provide generic guidelines without referring to ERROR. Later, I want you to ask me questions to practice what I learned about interpreting tracebacks.

Interpretation

After reading the suggested steps, we may try to interpret the traceback.

I understand the interpretation schema. Please lead me through this process of interpreting ERROR by asking questions in the suggested order. Please ask just one question at a time and wait for my response.

You can now go interactively with answering questions, asking for hints or answers. The model will by default offer you an explanation or solution, so you need to prevent it when asking just for a hint.

I do not have an idea. Could you give me a hint? Please do not answer the question, but just help me to find an answer on my own.

Alternatively, if it is too difficult for you, ask for a complete interpretation of the traceback and try the iterative process on another example.

Could you give me a traceback and practice with me interpreting it?

Practice till you achieve a desired fluency in interpreting error messages.

Fixing

Now, we may try to fix the problem. If it in the program we provided, we may start just straightforward, but if we asked for an example traceback, we may ask for corresponding code.

Could you give me an example code which could cause the error we just interpreted before we start fixing the problem? Please do not provide information on how to fix the problem. I want to do it on my own with your suggestions.

Next, you can go iteratively, answering questions, asking for hints or answers.

Ask for more examples and practice till you achieve the desired learning effect.

Exercise

Repeat this process with your own program, one from the Internet or ask the model to generate one for you. Introduce a modification to the program so it stops working. Alternatively, ask the model to generate a program with an error and a corresponding traceback.


Identifying Bugs

Motivation

It’s generally easier to fix a program that isn’t working at all than to diagnose and resolve issues in a program that is functioning incorrectly. Pinpointing exactly what’s wrong in a program that partially works can be more complex because the errors are less obvious. Developing the ability to effectively identify and addressing these subtler issues is a crucial skill in programming. However, mastering this skill requires patience and time, as it involves a deep understanding of how the program is supposed to function and where it may be going wrong.

Code

Let us introduce a mistake to a simplified version of the previous program by changing the comparison operator in the logical expression of the if statement. Instead of count > bigcount, we will use count < bigcount.

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]

counts = dict()

for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None

for word, count in list(counts.items()):
    if bigcount is None or count < bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

Role

Ask an LLM to play a role of your personal tutor adjusted to your skill level as desciribed at the begining of this tutorial. Just cpy the role prompt before you start woring on the actual task.

If an LLM starts to behave in an undesired way, you may go back to the calibration process to get a better role prompt. You may include it in the current chat or create a new chat to get a clean context.

Context

At first, we may provide the context as in the earlier sessions(s).

Please help me understand the given code by providing explanations at my current level. Please wait for my questions, in which I will refer to the given code as CODE.

Here is CODE:

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]

counts = dict()

for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

bigcount = None
bigword = None

for word, count in list(counts.items()):
    if bigcount is None or count < bigcount:
        bigword = word
        bigcount = count

print(bigword, bigcount)

Reporting a Problem

If we want to get a support in fixing bugs in our program from human or an LLM, we must specify what is the problem that we are facing. In our scenario, we may ask the model what is needed to report a problem.

My program is not working as I expected. Please let me know what kind of information I should provide so someone can help me solve the problem with CODE.

For simple programs and our scenario, we may just focus on expected and actual behavior.

Fixing the Problem

Now, we want the model to guide us to solve a problem.

I expected to get the most frequent word, but I got another word. Please give me a hint of what I should inspect to identify the source of the problem. I want to find out on my own, therefore, please do not provide any solution.

Once you get a hint, you may answer or ask for more details.

Could you cite the if-statement you are referring to and provide a more detailed hint?

You may continue till you solve the problem. However, in this example it is obvious as we introduced it ourselves. Moreover, if there is meaningful naming of variables in a program, it would be possible for the model to identify and fix the issue without any further details.

Experiment

We may experiment to see how naming of variables will impact bug fixing by the model.

Open a new session and ask for help providing the buggy program.

My program is not working correctly. Please help me to fix it.

Now, try the same in a new session, but with bigcount renamed to the_count and bigword to the_word.

text = [
  "It is easier to fix the problem with a program which is not working at all",
  "then to identify and fix a problem with a program which is working but incorrectly."
]

counts = dict()

for line in text:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) + 1

the_count = None
the_word = None

for word, count in list(counts.items()):
    if the_count is None or count < the_count:
        the_word = word
        the_count = count

print(the_word, the_count)

What is your conclusion?

Exercises

  • A basic task: Repeat this process with your own program, one from the Internet or ask the model to generate one for you. Introduce a modification to the program so it still works, but incorrectly. You may change logical expressions or calculations.

  • An advanced task: Repeat this process on a program that you really want to debug.

  • A reversed task: Ask an LLM to generate a program with errors or bugs and try fix it.


Solving Tasks

Motivation

While using a model to generate solutions can be convenient, it doesn’t contribute to improving your programming skills. True learning in programming comes from hands-on practice. If you’re aiming to become proficient, it’s essential to actively engage in writing programs yourself. This direct experience is key to understanding programming concepts deeply and developing the ability to solve problems independently.

Task

Let us a task from 198801 and the Python for Everbody textbook. To make it more complex, we merged two tasks into one: Chapter 3, Exercise 3; and Chapter 4, Exercise 7.

Write a program to prompt for a score between 0.0 and 1.0. If the score is out of range, print an error message. If the score is between 0.0 and 1.0, call a function determining a grade using the following table:

Score Grade
>= 0.9 A
>= 0.8 B
>= 0.7 C
>= 0.6 D
< 0.6 F

The function should be named compute_grade and take a score as its parameter and return a grade as a string. The program should not contain any user interaction, just the function.

Role

Ask an LLM to play a role of your personal tutor adjusted to your skill level as desciribed at the begining of this tutorial. Just cpy the role prompt before you start woring on the actual task.

If an LLM starts to behave in an undesired way, you may go back to the calibration process to get a better role prompt. You may include it in the current chat or create a new chat to get a clean context.

Context

As in the previous examples, we will set the context for our conversation first.

Please help me solve a task. I want to program it myself but I need hints that will help me to start. I am a novice programmer in Python, so guide me in small steps giving me just one hint at a time. Your hints should refer just to one statement needed to accomplish the next step towards a complete solution. I will respond with Python code expecting you to give me feedback.

If my solution is incorrect or incomplete, give me hints that will help me to fix the problem myself. Provide me with a bug fixed or improved version only if I explicitly ask you for it, otherwise just hints without any code.

In my questions, I will refer to the given task as TASK.

Here is TASK:

Write a program to prompt for a score between 0.0 and 1.0. If the score is out of range, print an error message. If the score is between 0.0 and 1.0, call a function determining a grade using the following table:

Score Grade
>= 0.9 A
>= 0.8 B
>= 0.7 C
>= 0.6 D
< 0.6 F

The function should be named compute_grade and take a score as its parameter and return a grade as a string. The program should not contain any user interaction, just the function.

Hints

After setting the context, you may start a conversation with the model. It will tend to provide you corrected version of your solutions, therefore you may need to remind the model about the tutoring rules from time to time.

Alternatives

When you complete the task, you may ask for another one to practice the same concepts. Depending on how easy the original task was for you, you may ask for a task at a different difficulty level (simpler/similar/harder). You may express wishes related to the syntax of the new task or domain.

An example, prompt could be this:

Now, I understand how to solve a TASK, but I would like to practice more. Please give me a similar task with the same data types and control flow statements. It should be slightly more difficult than TASK but with a lower number of cases in the if-statement. Generate a solution, but do not present it to me, just check if it is possible to solve the new task with the concepts I learned. I am studying physics and like mountain biking, so please provide something related to my interests.

If the generated task is not appealing, try to regenerate the response or modify your prompt.

If you want to learn only particular concept, e.g. logical expressions, you may also ask an LLM to generate a program when you need to fill gaps, e.g. expressions in if-statements.

Exercises

  • A basic task: Take any task you wish and repeat the proposed process. Continue till you achieve the desired learning effect.

  • A reversed task: Ask an LLM to generate a task for you.


Getting Help

Motivation

The official documentation is the most reliable source of information about a programming language. However, beginners might find the technical descriptions in these documents challenging to understand. In such cases, using a model can be beneficial. It can help you interpret the documentation and provide guidance on how to apply what you’ve learned. This approach allows you to gradually build your understanding while ensuring you have access to accurate and up-to-date information.

If you know a function name, you may ask about its documentation and example usage.

Please help me to understand how the print() function works in Python. Please give a description and a few (2-3) usage scenarios suitable for my level. Please keep your response short, but propose a few directions I could explore and wait for my decision to provide more details or examples for them.

Similarly, you may ask about a package.

I want to use the math package from Python. Please help me to learn about it. Please give a description and a few (2-3) most used functions with their short (1 sentence) explanation to comprehend at my current level.
Please keep your response short, but suggest what else would be useful to learn about this package and wait for my decision to provide more details.

If you use an LLM to get information about functions or packages, it is good to be aware of the cut off data of particular model. It can only provide information about the versions that were available when the LLM was trained. You may also explicitly ask the LLM to search for newest versions/options in the Internet.

Reading documentation

Reading technical documentation is a desired skill, as it is a reliable source of information providing precise and relevant descriptions. You may use a model to help you in learning how to interpret it.

I want to learn how to read technical documentation for Python. Please give me a docstring of one of the most frequently used functions in Python. Tutor me how to read it by giving me hints. Explain how to interpret pieces of information step by step. Next, ask me to interpret something; evaluate my response and give me hints if I got it wrong. Continue with next pieces of information till we finish interpreting the whole example.

Note the cut off data, see the Functions and Packages.

Apropos

Sometimes, you remember that there is a function or a package that you could use for your task, but can not recall its name. In R, there is apropos() function, that searches over the documentation and provides related functions. We may use a model to get similar results.

I would like to get a random integer. What function could I use for it in Python?

Note the cut off data, see the Functions and Packages.

Brainstorming

You may also ask a model to give you advice on what to use in a particular situation.

I would like to store and process data from a small excel file. My program will be executed on a slow computer with limited memory. Which Python package would you recommend to use and why? Please keep your answer short and clear for my current level. Do not give any example usage yet.

Note the cut off data, see the Functions and Packages.