My original intent was to “blog along” with the VB course that Andy Lattal is currently teaching at WVU. But, I started out behind and have only gotten further behind.
What happened? Part of the problem is my tendency to be interested in too many things at once. I have occasional periods of higher than usual energy, and start several projects at once. But then I invariably regress to my normal low energy state and begin neglecting some of my projects. And at age 71, those low energy states are much lower than they used to be (and perhaps I am a Covid “long hauler”, but who knows). That is part of what is going on now.
Traditional Linguistic Processing
But the real issue is that I am approaching VB as a computational problem of a particular sort. That is, where and how do I start producing Computational Verbal Behavior (CVB) software that is as useful as the Natural Language Processing (NLP) software that we all use and love?
I want to be able to do a search for CVB software and find packages more capable than that of traditional NLP, and having more fidelity to the human behavioral processes we call “verbal behavior”. How do we get there? Where do we start?
So far, the starting point seems to be to use traditional NLP as a set of tools to bootstrap CVB. This seems to be the approach of several very knowledgeable and people, such as Bill Hutchison, Ken Stephens, David Cox.
This is software that more or less does the same types of things we all learned to do in elementary school. Break text down into sentences, break sentences down into words and phrases, tag words and phrases as to grammatical units, diagram relationships among them, and so on.
Software such as:
- spaCy: A popular Python language NLP library known for speed, but a limited selection of algorithms, albeit generally the algorithms that are considered best of breed. See: When should I use spaCy?
- NLTK: Another popular Python NLP library. Its claim to fame is its wide selection of algorithms, though it is reputed to be a bit slower than spaCy.
I have used both, and sometimes used them together. See: Natural Language Processing: NLTK Vs SpaCy.
One very popular tool that is built on top of NLP is automatic classification or clustering of text. Arguably, the most popular tool for this is the open source Python library Gensim (short for “generate similarity”). What it does has come to be known as topic modeling.
David Cox recently gave an excellent webinar. Great overview of how one might use traditional NLP in support of more behavior analytic analyses of verbal behavior: Natural Language Processing: Quantitative and Computational Analyses of Verbal Behavior. Zoom indicated that the session was being recorded. I have a query in to the organization asking how (and at what price) I might obtain a copy (if that is even possible). I’ll update this post when I hear back.
My little story with regard to unsupervised auto-classification starts in the early 1990s when I worked for a search engine company whose primary algorithms were based on the work of Gerard Salton. In fact, the founder of the company was one of Salton’s PhD students. Here is all that remains of the company and the search engine, just a few pages in the Wayback Machine: Callable Personal Librarian (CPL).
Salton is known as “the father of information retrieval” for his development of the vector space model, which is still used in may derivations today in search engines, automatic category/topic discovery, etc. The Rocchio classifier is one such derivation. Latent Dirichlet allocation (LDA) is another such derivation. Both use the vector space model for classification.
When I had a need for auto-classification of text, my first thought was to contact my former colleagues at the search engine company for ideas. At the time the company was acquired by a large corporation and disassembled, the big brains were beginning to work on auto-classification using Salton’s vector space model (as was Salton, though apparently independently). Those events happened a couple of decades earlier, so for background information I did an online search first and came up with more than just background information.
Besides discovering that the LDA algorithm was a popular means of accomplishing automatic classification, I discovered a couple of embodiments of LDA, of which Gensim was the most popular (and most freely licensed). I also ran across a couple of interesting articles.
Here is a PhD dissertation that discusses LDA as an information retrieval method. It even cites Gerard Salton:
Here is the cited article:
- Salton, G., Wong, A. and Yang, C. S. (1975). A vector space model for automatic indexing. Commun. ACM 18 613–620.
Anyway, in the course of playing with Gensim I naturally ended up using spaCy and NLTK.
My Latest Stumbling Block
I’m reading along in the second article in Andy’s syllabus:
- Palmer, D. C. (2000). Chomsky’s nativism reconsidered. The Analysis of Verbal Behavior, 17, 51-56.
I run across this passage:
- “The reader may notice the kinship of this principle with Skinner’s concept of autoclitic frames …”
It has been 40+ years since I read Verbal Behavior, so to satisfy my WTF moment, I search online for what this most fundamental functional unit, “autoclitic”, might be. How does it function, and how might I model it in software? How might I discover autoclitic units in text, what would the code have to do to pick it out? How much context (stimulus environment) must be considered? Topography? Consequences?
OOPS: autoclitic is not a fundamental unit. It serves to set the occasion for slightly different responses than would a more fundamental unit with which it is associated. It modifies the stimulus environment in which the more fundamental unit is emitted, and thereby modifies the stimulus function by “compounding” it a bit more, by conditionalizing it a bit more.
Okay, now recurse. What are the other VB units that whose function may be changed by autoclitics? Here is a starting point:
So now I must recurse down the “knowledge tree” to find a leaf node with which to start.
Time to backtrack (a.k.a., “pop the stack”) and take a different tack? Maybe. But I think right now it is time for breakfast before a busy day.