# Ben Congdon - On Prompt Engineering (Highlights) ![rw-book-cover|256](https://benjamincongdon.me/blog/2023/02/18/On-Prompt-Engineering/cover.jpg) ## Metadata **Review**:: [readwise.io](https://readwise.io/bookreview/26288895) **Source**:: #from/readwise #from/reader **Zettel**:: #zettel/fleeting **Status**:: #x **Authors**:: [[Ben Congdon]] **Full Title**:: On Prompt Engineering **Category**:: #articles #readwise/articles **Category Icon**:: 📰 **URL**:: [benjamincongdon.me](https://benjamincongdon.me/blog/2023/02/18/On-Prompt-Engineering) **Host**:: [[benjamincongdon.me]] **Highlighted**:: [[2023-04-25]] **Created**:: [[2023-04-27]] ## Highlights - The release of the paper [*Large Language Models are Zero-Shot Reasoners*](https://arxiv.org/abs/2205.11916) showed how large of an influence prompt prefixes had on model reasoning performance. ([View Highlight](https://read.readwise.io/read/01gxpv8bm3xnjkzvnsjqtq1m1n)) ^507049534 - [Anthropic](https://www.anthropic.com/)’s [Constitutional AI](https://www.anthropic.com/constitutional.pdf) paper introduces a new usage of prompt engineering: in steering the top-level behavior of the model, by including specific prompts as “constitutional principles” during its fine-tuning training. ([View Highlight](https://read.readwise.io/read/01gxpv7fp5mfenbp44mrdhbycn)) ^507049378 - OpenAI undertook a similar project in fine-tuning GPT-3.5 into [ChatGPT](https://en.wikipedia.org/wiki/ChatGPT) using *Reinforcement Learning from Human Feedback* ([RLHF](https://huggingface.co/blog/rlhf)). With RLHF, you take a pre-trained LLM and expose it to “reward” or “punishment” based on human-labeled judgement of its output. Among other effects, this process nudges the model towards behaving more conversationally (e.g. “helpfully”). ([View Highlight](https://read.readwise.io/read/01gxpv6sgt5hvtxsf3ey3bgtny)) ^507049178 - During the fine-tuning phase, the fine-tuned LLM is generates 2 outputs. These outputs are fed to the PM, which selects its choice of the less harmful output. This choice is then fed back to the LLM as the reinforcement learning signal, which nudges the model into producing outputs more like the PM’s preferences over time. ([View Highlight](https://read.readwise.io/read/01gxpvb8mdtd3n3k82ptqbw742)) ^507049754 - The authors note that if they skip the “critique” phase entirely, and have the LLM just produce a revised version with respect to one of the principles, they achieve effectively identical harmlessness scores ([View Highlight](https://read.readwise.io/read/01gxpvetpv43axby4hreh69esc)) ^507050317 - If you want to look at (purportedly) real prompts of in-production AI systems, researchers have helpfully leaked[1](https://benjamincongdon.me/blog/2023/02/18/On-Prompt-Engineering/#fn:1) the [Perplexity.ai](https://news.ycombinator.com/item?id=34482318)[2](https://benjamincongdon.me/blog/2023/02/18/On-Prompt-Engineering/#fn:2) and [ChatGPT Bing](https://news.ycombinator.com/item?id=34777646) prompts! ([View Highlight](https://read.readwise.io/read/01gxpvkgz3c7asnmx0pq3d4rb5)) ^507051222 - Microsoft released [prompt-engine](https://github.com/microsoft/prompt-engine) in late 2022, and it [made the rounds](https://news.ycombinator.com/item?id=34811070) again this week, sparking some interesting discussion. As I understand it, it’s effectively a library for building and maintaining a prompt as a user interacts with an LLM. ([View Highlight](https://read.readwise.io/read/01gxpvtdjn49gsndmdfvrahva3)) ^507053194