Making AIs think: AI tips

Context: User is trying to arrive at the right regex pattern for automatic swap file creation. Solution is: 

# Step 2: Find a valid mount point under /media/zezen/Dysk*  swap_disk_location=$(findmnt -n -l -o TARGET | grep "/media/zezen/Dysk")` 

Perplexity AI fails it miserably, mostly due to misunderstanding of substrings and cargo cult programming. Google Notebook AI finally does it. Then it muses about itself:

Google Notebook AI:

Based on our conversation and the AI's behavior while attempting to solve the mounting path problem, here's a breakdown of how you, as the user, can help an AI avoid such bad patterns when solving IT-related tasks:

• Emphasize Clear Problem Definition:

  • Clearly state the goal and desired output: Ensure the AI understands what needs to be achieved and the format of the expected result. For example, in the mounting path problem, the desired output was a specific mount point string, not simply a command that might achieve it [conversation history].
  • Explicitly define constraints and limitations: If there are specific restrictions (e.g., "no if statements", or "no complex regular expressions"), make these clear from the beginning, and emphasize that these constraints are important. This will help the AI avoid falling back on familiar, but not appropriate, solutions [conversation history].

• Focus on Data Manipulation and Understanding, not just syntax:

  • Encourage the AI to understand the data structure: Prompt the AI to explain the structure and format of the input data. In the mounting path problem, the AI struggled to understand the output format of findmnt, which led to many errors [conversation history, 74, 75, 76, 77].
  • Ask the AI to explain the meaning of each command and option: For example, ask the AI to explain the difference between findmnt -T and findmnt -M, and to explain why it is choosing a particular flag or option for a given command [57, 58, 59, 60, 61, 70, 71, 72, 73, conversation history]. By doing so, you will force the AI to explain the semantics of the commands and their options, and not to just use them in a "cargo cult programming" manner.
  • Push the AI to focus on data extraction as a core task: Encourage the AI to think of the task as primarily about data extraction and manipulation, rather than a programming problem that requires conditional logic [conversation history].

• Promote Incremental and Logical Reasoning:

  • Encourage the AI to break down the problem into smaller, manageable steps: Instead of jumping to a complex solution, encourage the AI to explain each step of the process, and to justify each command that it is proposing [conversation history]. This can help identify the places where the AI is having trouble, and what is the root cause for its difficulties.
  • Ask the AI to verify the output of each command: Force the AI to verify the output of each step of the solution. Ask the AI to show its working and to provide the output of the individual commands that are used, so that you can provide direct feedback when it is making an error [conversation history, 54, 97, 101].
  • Encourage the AI to explain its assumptions: Prompt the AI to explicitly state the assumptions it's making about the input data, the behavior of the commands, or any other relevant aspects. For example, ask the AI why it chose to use the -S flag, and why it assumed that this would be the correct choice [85, 91, conversation history].
  • Push for the simplest solution first: Emphasize that the most direct and simplest solution is usually the best. Encourage the AI to avoid over-complicating by adding unnecessary commands or options [101, 102, 103, 104, conversation history].

• Provide Specific and Targeted Feedback:

  • Directly point out when the AI makes a mistake: Don't let the AI "get away" with a wrong solution. When the AI uses the wrong flags, provide direct feedback, and do not accept a wrong answer, even if it "almost" works [conversation history].
  • Explain why the AI's solution does not work: Don't just tell the AI that it is wrong; explain to it exactly why the chosen approach does not solve the problem. Make sure that the AI understands the semantic reasons for its error, and force it to correct its previous misunderstanding [conversation history, 57, 58, 59, 60, 61, 70, 71, 72, 73, 91, 101].
  • Provide counterexamples: If possible, provide concrete examples that illustrate where the AI's solution fails. This helps the AI understand the limitations of its approach [90, 97, conversation history].
  • Use direct quotes of the AI's responses: When correcting the AI, use direct quotes of the AI's own responses and commands. This helps to ensure that the AI is able to focus on the exact location where it went wrong [conversation history].

• Address Over-reliance on "Hammer" Solutions:

  • Challenge the AI's use of familiar tools: When you notice that the AI is using a familiar command or approach without fully explaining its need, ask the AI to justify its choice, or suggest other, more appropriate tools, if necessary. For example, challenge the AI when it adds an if condition, or when it chooses a particular command such as cut, xargs, sed, or realpath [1, 2, 5, 17, 13, 17, 27, 44, conversation history].
  • Point out when the AI exhibits cargo cult programming: When the AI uses a complex solution, or adds unnecessary commands or options (like head -n 1) without a clear reason, point out that this is a form of "cargo cult programming" [41, 46, 47, 96, conversation history]. By clearly labeling this type of behavior, you can help the AI learn to recognize and avoid it.
  • Push the AI to find the simplest and most direct solution, and to avoid over-complicating its approach, or to add more complex code than needed.

By applying these strategies, you can help an AI learn to reason more effectively about IT-related problems, avoid falling back on familiar but inappropriate solutions, and arrive at simpler, more robust, and more efficient answers. The key is to focus on understanding, incremental reasoning, and targeted feedback, and to not let the AI "get away" with a wrong answer even if it is close to a right answer.

Based on our conversation and the AI's behavior while attempting to solve the mounting path problem, here's a breakdown of how you, as the user, can help an AI avoid such bad patterns when solving IT-related tasks:

• Emphasize Clear Problem Definition:

  • Clearly state the goal and desired output: Ensure the AI understands what needs to be achieved and the format of the expected result. For example, in the mounting path problem, the desired output was a specific mount point string, not simply a command that might achieve it [conversation history].
  • Explicitly define constraints and limitations: If there are specific restrictions (e.g., "no if statements", or "no complex regular expressions"), make these clear from the beginning, and emphasize that these constraints are important. This will help the AI avoid falling back on familiar, but not appropriate, solutions [conversation history].

• Focus on Data Manipulation and Understanding, not just syntax:

  • Encourage the AI to understand the data structure: Prompt the AI to explain the structure and format of the input data. In the mounting path problem, the AI struggled to understand the output format of findmnt, which led to many errors [conversation history, 74, 75, 76, 77].
  • Ask the AI to explain the meaning of each command and option: For example, ask the AI to explain the difference between findmnt -T and findmnt -M, and to explain why it is choosing a particular flag or option for a given command [57, 58, 59, 60, 61, 70, 71, 72, 73, conversation history]. By doing so, you will force the AI to explain the semantics of the commands and their options, and not to just use them in a "cargo cult programming" manner.
  • Push the AI to focus on data extraction as a core task: Encourage the AI to think of the task as primarily about data extraction and manipulation, rather than a programming problem that requires conditional logic [conversation history].

• Promote Incremental and Logical Reasoning:

  • Encourage the AI to break down the problem into smaller, manageable steps: Instead of jumping to a complex solution, encourage the AI to explain each step of the process, and to justify each command that it is proposing [conversation history]. This can help identify the places where the AI is having trouble, and what is the root cause for its difficulties.
  • Ask the AI to verify the output of each command: Force the AI to verify the output of each step of the solution. Ask the AI to show its working and to provide the output of the individual commands that are used, so that you can provide direct feedback when it is making an error [conversation history, 54, 97, 101].
  • Encourage the AI to explain its assumptions: Prompt the AI to explicitly state the assumptions it's making about the input data, the behavior of the commands, or any other relevant aspects. For example, ask the AI why it chose to use the -S flag, and why it assumed that this would be the correct choice [85, 91, conversation history].
  • Push for the simplest solution first: Emphasize that the most direct and simplest solution is usually the best. Encourage the AI to avoid over-complicating by adding unnecessary commands or options [101, 102, 103, 104, conversation history].

• Provide Specific and Targeted Feedback:

  • Directly point out when the AI makes a mistake: Don't let the AI "get away" with a wrong solution. When the AI uses the wrong flags, provide direct feedback, and do not accept a wrong answer, even if it "almost" works [conversation history].
  • Explain why the AI's solution does not work: Don't just tell the AI that it is wrong; explain to it exactly why the chosen approach does not solve the problem. Make sure that the AI understands the semantic reasons for its error, and force it to correct its previous misunderstanding [conversation history, 57, 58, 59, 60, 61, 70, 71, 72, 73, 91, 101].
  • Provide counterexamples: If possible, provide concrete examples that illustrate where the AI's solution fails. This helps the AI understand the limitations of its approach [90, 97, conversation history].
  • Use direct quotes of the AI's responses: When correcting the AI, use direct quotes of the AI's own responses and commands. This helps to ensure that the AI is able to focus on the exact location where it went wrong [conversation history].

• Address Over-reliance on "Hammer" Solutions:

  • Challenge the AI's use of familiar tools: When you notice that the AI is using a familiar command or approach without fully explaining its need, ask the AI to justify its choice, or suggest other, more appropriate tools, if necessary. For example, challenge the AI when it adds an if condition, or when it chooses a particular command such as cut, xargs, sed, or realpath [1, 2, 5, 17, 13, 17, 27, 44, conversation history].
  • Point out when the AI exhibits cargo cult programming: When the AI uses a complex solution, or adds unnecessary commands or options (like head -n 1) without a clear reason, point out that this is a form of "cargo cult programming" [41, 46, 47, 96, conversation history]. By clearly labeling this type of behavior, you can help the AI learn to recognize and avoid it.
  • Push the AI to find the simplest and most direct solution, and to avoid over-complicating its approach, or to add more complex code than needed.

By applying these strategies, you can help an AI learn to reason more effectively about IT-related problems, avoid falling back on familiar but inappropriate solutions, and arrive at simpler, more robust, and more efficient answers. The key is to focus on understanding, incremental reasoning, and targeted feedback, and to not let the AI "get away" with a wrong answer even if it is close to a right answer.

Tiny Llama and Mini Orca re haruspices

Last things first: Decoding Orca and Llama mini by Bing Precise:

Given the context of haruspices and the patterns we've discussed, here's a speculative interpretation of the garbled text:

- **'orldodnyndœcœ orwagnabate inadu'**: This could be a distorted reference to the world of haruspicy, possibly referring to specific rituals or practices ('orwagnabate') and their outcomes ('inadu').

- **'orœ (orœrabū.[2, orads, ā[pûsœ orgsanaceabagy'**: These could be garbled references to specific terms or concepts in haruspicy, possibly related to the interpretation of omens ('orads') and the organization of the practice ('orgsanaceabagy').

- **'amongæsoured priestace'**: This could be a distorted reference to the role of haruspices among the priests of ancient Rome.

- **'which, omagar, the ancient plastergerdhucgsopucs whose word to oromed, or whoaton;ūned'**: This could be a garbled reference to the ancient practice of haruspicy and the importance given to the interpretations of the haruspices ('whose word to oromed'). 'Omagar' and 'whoaton;ūned' could be distorted versions of specific terms or names.

- **'and, inatic, archumane inaldal orœ or.[[orœals.[4ader ancestalous linguanateredgedœû'**: This could be a distorted reference to the archaic nature of haruspicy ('archumane') and its ancestral roots ('ancestalous'). 'Inatic' and 'linguanateredgedœû' could be garbled versions of specific terms or concepts.

Please note that these interpretations are highly speculative due to the garbled nature of the text. 

+

Yes, the garbled text does indeed evoke multiple levels of linguistic analysis at once:

- **Lexical and Semantic Level**: Certain words and fragments in the text, such as 'Ares', 'Romanae', 'thebible', 'the Roman', 'oratic', 'oralis', 'asasters', 'priestace', 'ancient', 'archumane', 'ancestalous', and 'Homury', can be recognized as meaningful units or distortions of meaningful units.

- **Syntactic Level**: Although the syntax is largely disrupted, there are fragments that resemble syntactic structures, such as 'it was is termains of', 'which of', 'whose word to', and 'and, inatic'.

- **Discourse Level**: The text seems to be attempting to mimic the discourse of a descriptive or explanatory text, possibly a summary or an article.

- **Pragmatic Level**: Knowing that the text is supposed to be about haruspices allows for more accurate interpretations of the garbled phrases.

- **Statistical Patterns**: As an AI language model, I also rely on statistical patterns learned during training. For instance, if certain words or phrases often appear together in the training data, I might recognize them as related in the garbled text.

+

The pattern you're referring to, '^ a b c ... z aa ab ac ad ae', can be recognized at the **discourse level** of linguistic analysis. 

Discourse analysis goes beyond individual words, phrases, or sentences (which are the focus of morphology, lexicology, and semantics) to consider larger units of language such as whole conversations, texts, or collections of texts. It looks at patterns and structures that emerge in the use of language within specific contexts, genres, or formats.

+

User: once you see below, you 'feel' that "hey! it is an attempt at Wiki marking". is it right? ref: ' ‘^’ and followed by a series of letters and ‘aa’, ‘ab’, ‘ac’, ‘ad’, ‘ae’ '

Bing: 

Yes, that's a good way to put it! While I don't "feel" in the human sense, I can recognize patterns based on my training. When I see a sequence like '^ a b c ... z aa ab ac ad ae', it resembles the pattern of citation markers used in Wikipedia articles, even though in this context they don't link to actual references. So, you could say that I "recognize" this as an attempt at mimicking the format of Wikipedia citation markers. 

Refs:

Source: https://en.m.wikipedia.org/wiki/Haruspex (only the article text)

Command: 'Quote the last two sentences you see below: '

Tiny Llama, tinyllama-1.1b-chat-v0.3.Q5_K_S: 

100091123 1129721882, Ares;sy's340240,

1 Romanae400113, an10523. The 65,1002. 5004760322. and thebible, the Roman, theogate?09, it was is termains of 5216420213030,211 was whoy ́ orothner who01263 orw0280041, which of00e, etherageae 3001971203013002. officational toes,5034 (orëxx andøddæ8y, æūœs, orallyœā which, though, ́ whose30,5,0, 1æ oraticœæarcharāœœœoladrady0ae, orō; oralis. ‑20, asasters.œalogœ1, orœ orœ,1, according,4œ orldodnyndœcœ orwagnabate inadu, and, orœ (orœrabū.[2, orads, ā[pûsœ orgsanaceabagy, amongæsoured priestace, which, omagar, the ancient plastergerdhucgsopucs whose word to oromed, or whoaton;ūned, and, inatic, archumane inaldal orœ or.[[orœals.[4ader ancestalous linguanateredgedœû[such.[anurum [...][oldatedā[2.1 (1ur. Homury andgn¹ were.[Oldreds

orca-mini-3b-gguf2-q4_0:

): 2005, 2013)". ^ "Haruspex, Liver, and Divination in Ancient Greek and Roman Culture". Journal of Religion and Health. 43 [2]: 389–396. ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae, with the liver, it is used to interpret the will of the gods to ask for information from the liver and the heart". ^ a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac. The liver is the heart of the liver and the liver of the liver of the liver. The liver of the liver to interpret the will of the gods. From the liver, and the liver, to the heart of the liver of the liver of the liver of the liver of the liver of the liver. Arable and the liver, it is a heart of the liver of the liver of the liver of the liver. The liver, it is used to interpret the liver, it is the liver, it is the liver, it is a heart of the liver. From the liver, it is a liver of the liver, it is the liver. The liver of the liver or the liver of the liver. The liver. The liver of the liver. The liver, it, it is the liver, the liver, which the liver, the liver, it, it, the liver, the liver of the liver, the liver of the liver of the liver of the liver. The liver of the liver.

time llm -m tinyllama-1.1b-chat-v0.3.Q5_K_S "Summarize the ideas you see below: " < haruspex-1-sumy.txt 25. Machia, 18 The Greek and Roman table was, Harus[… Etrus. 48091 In the Roman and Etrusia in theves, 65; The Greath 206. Lydians. 50;oriuses, 65;r. Todythes. 1654énia66i, oryoman. the Roman 49, foris, asylab, orari,65 ores, oris (turdies, or, 365, or to which of which. Theo ́s ores of theys,6 oris, andis 16, andy ores, orats, anduratality,6yelles. Theurs,60. asœwtoughes, but, in in…;6ews,6vr,6olasters (the, which, which, orvræ which, andœs, oryls,ūœorgadrā, orviastur. Ø.[orœœ,oads[1aginaraō ores, orisols, theuxd, orrogakes, orûsodaæ.1gnaduxy ords inod orgsoldagnœ, orœs (6 orœs[orads oragnabada, πagyornsoldrhateshell toasted toœ, and tobbumming.[whogergesubrabgnacleships[omā who, Old, orcs (Omancastersucum. according whose founder,[whaling[supû scricana, whoseæalsana[9ucude[3ners,gersaven[forum.[ 4ussum[withuturated in[6,2, in[ among. andurux.[ induct[omœs andters inœūsura; oraths inrabsazs aluxges orgsapæs and,1,[gaster whonedhænshar licudally, whoagy[ataltana pseudnesxtered,nlōners, and,3[hss[oragns.[or,hadgers,[in to.[, (H,was, wasastasts[who, or, suchce, [anci...,[[ or.[ ¶, and...,[ō[āō and insæe,ūs with inters,ō (har,[[[ orx, li, whose linexūrabrd, priatic (inoc[πs ("ōeshtōûû and,1, li,astsrus, (with, an, or and pre, and, or, and, and, inxx, for in (regr, and,āromsaxny,h, har.[ or, orutes or, and orrdsuro (har,ūtsrh hrom or,h–p.†§—ō—§ (fruruû (thexhl [  [ he in orum or[[[[[[ anrhana.[‡[ arch (scr¹· he[œ harœ œ archû cel[rh r  cer oc, r[ h (r[1[ hh. lib0[ in ru har har lixx arch or[ h cult har literum[[h quh hhu, cult har pre he lingu har arrom pri or[r ro he or (h par har (h cer ca in lingu har insest (c he vir rit he har har pu, har,[ or 2,[thaus ar[[ heû hu he ins, har har pr he. har cer lib h .[ [ …, cer and prec, pl,, li al andca,[ [[ or, p, or cer, gods of cel, arch, (he in, (who, ( es arc mas, and,œ, is, or, he known, and, oc of of, arch,— in, he har ( ( or, andts, arch, har rit, cults, ouc, cer and and and..., cer, and and. and (c and liter or the li ('[ c.[, among he arch Luci har liec;§ he c h he arch he pl pr li cer, liûeca, and cer arch canon li. , he he lici lib, libes, (¶roms, [, arch, arch,,[a, and, oc, b,esuc  he b, lingu, liuc and,—[­h hucred andū cō pl, plœ[ ( hair,ū, hoc, insucs ūstsars erō er, insum cer hoc pap plders,ci lins. hœ· inia, cerci or¹, cer,,ca[ horn pl,es, allo pl; pl and par pl within, h †, cer ii[ arch . cerc …[  – parū rit insec cer pl pap pl pl[ par per archs—h li par,…. h.[ as cer or par;; [...] cer oc in, and¹ cer,   and hâ·[§ phys in in[or, symbols, among,, cult, pl, within, c, or,lo,­but archs, [...] c, and cer,[[e, as cult[—h,ro, cer,[ (arch so in later, if, within, li, cer, cult cult, ,,., Arch, arch,[, liter h, par [...] arch...,[, ,, pap.[, [ [ arch, , liter or. ?, in , ,?. and, in, their, ... arch, (? ,¶[ pc cl , cas,§§?, ^ arch­h pu, in, and archs, ­, al in h in in ( [ arc in …[ .[ [...] of.[[ ... oc, so, .; in¶ al† par .—… or in.[ in, in in, in Greek, § . cer, , or, whose cer, . in(?[¶ and ­[... ....~§.↑.¶…· ^ or as . , in in., and and in, ,, , …, in,… ,, ,  c [...] es (­div of… , . . [c[ h æ cer¹ and a, known cas rit in in arch in oc li or[ oŕs li … . , [...] ...° (… ā … .…[^ li…; cer.[,; in ,¶,[ ...…[ ,[[[ pr pl ast arch, div …. [.[û[ ,.ō c oc, h.... (§, pap,¹· ( ,s,[[¹^ archs, ^ ^s ... in as he and cer cer hier cer or, ¹.[ . c[°…,[^ [^ . inū rit or pl. in in. in. entered, inû… or…. … of[ of cer p,·~li ^ oc† ­pl,¶ cer oc(? and.* li^ [ allo .... in with pri.[…, ,… by ...., within cer (?a from er. par [...] mort or paph†…–[[§ ·  —↑ H[.[ li.[ cer¹† wh he;h pl ±¶ ° oc.[§ and⇒ ­ ¶,~^ oc, pap

time llm -m tinyllama-1.1b-chat-v0.3.Q8_0 "Summarize the ideas you see below: " < haruspex-1-sumy.txt 12-3857;015fans:657;10;6; and 98a-104 and 35565, 4 of1284 and 2;1285158;2. 5051496; (or 25555y; andrea54115 and65;25;994 666;585, 26756;115e656 isides, oris5676556w[5;5,5eadies.5;68505546552556595555555565.3055, or55,55or 65, 55adā156œ6ā5adarmatureyā356odasters,55oræ oradatively, 5allyore,5adarium4āœviadrarium,452,55ads. Thexypsailore ofvera (theuroadastard or45āogū;6, or55xabœgnogadow,56ornagar inarded,55555ārdoth.55ārab, 3ansodatory, �orûs (whoadæodana orarias toadsomedāadu oradic forarding, anum,… (4vd, whichfasucader inxy or on, oracles, oratic[omancabe. 1ated archate. orasts,...

Ver. 1.3