LLM Quiz - Questions & Answers

What practice would help reduce hallucinations in an LLM giving factual advice?

  • Using open-ended questions
  • Providing specific source references in prompts ✅
  • Allowing the model to guess the missing data
  • Encouraging speculative responses

Which of the following strategies is least effective in reducing hallucinations in language models?

  • Reinforcement learning from human feedback (RLHF)
  • Using a smaller dataset for training ✅
  • Fine-tuning on domain-specific data
  • Incorporating factual consistency checks

Which of the following statements is true about the licensing of open-source LLMs?

  • Open-source LLMs cannot be used for commercial purposes
  • All open-source LLMs must be distributed under the Apache License
  • Open-source LLMs can have a variety of licenses, some of which may impose specific usage restrictions ✅
  • Open-source LLMs always come with no usage restrictions

How does a high temperature value affect the probability distribution of the next token in LLM outputs?

  • It sharpens the distribution, making high-probability tokens more likely
  • It flattens the distribution, making low-probability tokens more likely ✅
  • It has no effect on the distribution.
  • It biases the model towards shorter responses

Which scenario best exemplifies the use of one-shot prompting?

  • Providing a detailed list of instructions followed by multiple examples
  • Giving one example of a complex task and expecting the model to generalize ✅
  • Using a large dataset to train the model incrementally
  • Setting up a reinforcement learning environment with numerous iterations

What is the primary issue with the "bias amplification" phenomenon in AI systems?

  • It causes AI models to underperform in terms of accuracy
  • It leads to the reinforcement and exaggeration of existing biases in the data ✅
  • It makes AI systems more sensitive to noise in the input data
  • It results in overfitting the training data

In one-shot prompting, the primary goal is to

  • Train a model from scratch with a single data point
  • Fine-tune a pre-trained model using a single example
  • Generate a desired response from a model with one example in the prompt ✅
  • Use multiple examples to improve model accuracy

In what ways can the efficacy of prompts in multilingual models be improved?

  • Applying language-specific nuances ✅
  • Using translation tools ✅
  • Avoiding cultural references
  • Providing examples in multiple languages ✅

In a customer recommendation system, how can hallucination errors be minimized?

  • Process real customer feedback in updates ✅
  • Ensure realistic constraints in prompts ✅
  • Provide probabilistic confidence scores ✅
  • Ignore rare and unique customer behavior patterns

What is a potential drawback of few-shot prompting that practitioners should be aware of?

  • High computational costs during inference
  • Lack of flexibility in adapting to new tasks
  • Risk of overfitting to the examples in the prompt ✅
  • Requirement of large-scale pre-training datasets

Which of the following is a key difference between the development communities of open-source and closed-source LLMs?

  • Open-source communities typically involve contributions from a wide range of independent developers and organizations ✅
  • Closed-source communities often have more diverse contributions from various stakeholders
  • Open-source communities do not allow any external contributions
  • Closed-source communities are known for having more transparent development processes

Which of the following strategies is most effective for reducing the length of responses generated by a language model without significantly compromising on the quality of the response?

  • Reducing the temperature parameter.
  • Using more specific and detailed prompts
  • Setting a maximum token limit for the response ✅
  • Increasing the batch size during inference

In the context of preventing hallucinations in generative AI models, what does "model distillation" refer to?

  • Reducing the model size by approximating a larger model ✅
  • Using distilled water to cool down AI hardware
  • Training a model on distilled facts to ensure accuracy
  • Distilling the model's training process to essential components only

In the context of fine-tuning an LLM for a specific application, why might one opt to use a lower temperature setting during inference?

  • To encourage the generation of highly diverse outputs
  • To enhance the randomness and creativity of the model
  • To reduce hallucinations and generate more precise responses ✅
  • To increase the likelihood of generating factual inaccuracies

What steps can be taken to ensure LLMs provide culturally sensitive outputs?

  • Curate culture-specific datasets with diverse perspectives ✅
  • Employ region-specific context in prompts ✅
  • Use a single culture's dataset to maintain consistency
  • Validate outputs by cultural experts ✅

A generative AI used for educational content sometimes includes outdated information. What methods can address this?

  • Regular updates with the latest academic research ✅
  • Encouraging model creativity over factual accuracy
  • Cross-verifying outputs with up-to-date references ✅
  • Incorporate a feedback mechanism for educators ✅

Which of the following scenarios would most benefit from using a higher temperature setting for an LLM?

  • Summarizing legal documents
  • Generating poetry or creative writing ✅
  • Answering factual questions
  • Translating technical manuals

When optimizing prompts for generating structured outputs (like JSON), which of the following modifications can significantly improve the model's accuracy in producing the desired structure?

  • Adding explicit instructions to the prompt. ✅
  • Training the model on a smaller dataset with similar structures
  • Using a higher learning rate during training
  • Reducing the model's context window

Which of the following approaches is best suited for optimizing prompts to ensure that a language model generates responses that are both concise and contextually relevant?

  • Increasing the model's attention span
  • Incorporating keywords related to the desired context
  • Iteratively refining the prompt based on generated outputs ✅
  • Increasing the number of training epochs

How can developers ensure generative AI avoids spreading misinformation?

  • Using current and reliable sources ✅
  • Regularly updating the model with new training data ✅
  • Implementing cross-referencing mechanisms within the model ✅
  • Encouraging creativity over accuracy