In the previous tutorials we already met the most important newLISP data type, the list. A list is technically spoken a sequence of one or more elements, separated by one space and surrounded by balanced parentheses (notice that an empty list exists in newLISP also). Remember:

The parentheses around a list identify a unit, which can store data but also has the means for calling functions and defining functions.

The latter option ‘means for defining functions’ will be discussed in another tutorial.

Some examples:

;a list of numeric atoms, or more briefly, numbers, having five elements
(1 2 3 4 5)

;a list of three string elements
(“Reinier” “newLISP” “VisualNEO”)

;a list containing one string
(“this-is-the-only-string-in-this-list”)

;a list of symbolic atoms, or more briefly, symbols, having four elements
(FOO A1$2 I-AM-A-SYMBOL BWV545)

Symbols can serve as e.g. a variable or a function name (see below).

A list can combine different kinds of elements.

;a list with numbers, strings and symbols, having seven elements
(5 “my-number” A B “c” 3.14 PI)

A list can also be an element of a list.

;a list with a “nested” list
(1 (A1 B2,4 Hello! z-score) “2”)

Notice that this list has three elements. To be sure:

(length (1 (A1 B2,4 Hello! z-score) “2”))
→ 3

The second element, the sublist

(A1 B2,4 Hello! z-score)

has four elements. Notice that the space is the default separator within a list, so

– B2,4 is one element, despite its comma
– Hello! is one element, despite its exclamation mark.
– and z-score is one element, despite the hyphen-minus sign.

Nested lists make the so called ‘association lists’ possible, which are very useful: see Section 5.

The elements of a list in my newLISP-tutorials till now are numbers, strings, symbols and lists. The question arises, how does newLISP evaluate these data types? Study the next examples:

;Reinier is the son of Balten
(son Balten Reinier)
;Reinier is the father of Daniel
(father Daniel Reinier)

Here our little database have only symbols. It provides information about family relations: Balten is the grandfather of Daniel. But these items do not have any meaning to newLISP: they evaluate to themselves -are taken literally- just like numbers and strings.

(add 1.5 2.6)
→ 4.1

In this case, newLISP knows the symbol ‘add’ as a function and interpretes the two numbers 1.5 and 2.6 as its arguments. The expression will be evaluated, meaning: processing the function ‘add’ with two arguments and returning a value (here the sum between the two numbers 1.5 and 2.6).

(setq x 12)
→ 12

(+ x 13)
→ 25

Here is the symbol ‘x’ a variable; the value 12 is assigned to ‘x’. Then, 13 is added to ‘x’, which results in the sum 25.

So more precisely, a list can contain atoms (e.g. numbers, strings and symbols) and/or lists. Symbols can have the role of identifiers, naming variables or functions. For now, this is sufficient for you to know.

A small note on strings: strings and lists belong to the sequence data type (=ordered set of elements). They have many functions in common.

Let’s have a look now how to process lists. I’ve made again examples of easy newLISP-expressions. But believe me, lists lend itself to very complex constructs.

Section 1. List processing

1.1 Return the first list element

(first ‘(1 2 3 4 5))
→ 1

Notice that if a list is a argument of a function (here the function ‘first’), it needs to be quoted (= taken literally): see section 2.

(first ‘((1 2 3) 4 5))
→ (1 2 3)

1.2 Return the last list element

(last ‘(1 2 3 4 5))
→ 5

1.3 Return all elements of a list, except for the first.

(rest ‘(1 2 3 4 5))
→ (2 3 4 5)

Notice that the return value is a list.

(first (rest ‘(1 2 3 4 5)))
→ 2

1.4. Return values via implicit indexing.

(setq mc-list ‘(1 2 3 4 5))
→ (1 2 3 4 5)

(1 mc-list)
→ (2 3 4 5)

which is equivalent to the function ‘rest’.

(2 mc-list)
→ (3 4 5)

which is equivalent to (rest (rest ‘(1 2 3 4 5)))

(0 1 mc-list)
→ (1)

(2 -1 mc-list)
→ (3 4)

1.5. Nested list

(setq mc-list ‘(1 (2 3) 4 5))
→ (1 (2 3) 4 5)

This list has 4 elements. The second element is:

(1 1 mc-list)
→ ((2 3))

The fourth element is:

(3 1 mc-list)
→ (5)

The fifth element is the end of the list:

(4 1 mc-list)
→ ()

Yes, an empty list.

1.6 Modifying a list

1.6.1 ‘setf’

With the function ‘setf’, you can change characters of a list (string or array) by their index numbers.

;define a variable with ‘setq’
(setq my-list ‘(“a” “b” “c” “a”))

;replace the first “a” (index number 0) with “D”
(setf (my-list 0) “D”)
→ “D”

;the original list has been changed
my-list
→ (“D” “b” “c” “a”)

See tutorial 3, 1.6.1 if you don’t want to modify the original list definitively.

1.6.2 set-ref, set-ref-all and replace

(setf (my-list 0) “D”)

has an alternative:

(set-ref “a” my-list “D”)

which replaces the first match.

To replace all matches you could you use ‘set-ref-all’.

(setq my-list ‘(“a” “b” “c” “a”))
→ (“a” “b” “c” “a”)

(set-ref-all “a” my-list “D”)
→ “D”

my-list
→ (“D” “b” “c” “D”)

Of course, the function ‘replace’ does its job on lists very well. See www.newlisp.org/downloads/manual_frame.html

1.6.3 ‘push’ and ‘ pop’

The function ‘push’ adds a new element at the beginning of a list if no index number is specified. In the next example “X” is added to the end of a the list because the index number -1 is given.

(setq my-list ‘(“a” “b” “c” “a”))
→ (“a” “b” “c” “a”)

(push “X” my-list -1)
→ (“a” “b” “c” “a” “X”)

In the next example the index number 1 refers to the second position of the list, where “Y” is inserted:

(push “Y” my-list 1)
→ (“a” “Y” “b” “c” “a” “X”)

The function ‘pop’ removes an element of the list:

(pop my-list 0) ;the same as the default (pop mylist)
→ “a”

my-list
→ (“Y” “b” “c” “a” “X”)

1.7. Convert list to string

To convert a list to a string, you can use the function ‘format’:

(format “%d %d %d” ‘(1 2 3))
“1 2 3”

(format “%f %d %f” ‘(1.1 2 3.2))
“1.100000 2 3.200000”

or better

(format “%0.2f %d %0.1f” ‘(1.1 2 3.2))
“1.10 2 3.2”

(format “%0.2e %d %0.1f” ‘(1.1 2 3.2))
“1.10e+00 2 3.2”

(format “%s %s %s” ‘(“a” “b” “c”))
“a b c”

(format “%c %c %c” ‘(65 66 67))
“A B C”

where the digits refer to ASCII values in decimal.

The % (percent sign) in the previous examples starts a format specification: see table below.

format	description
s	text string
c	character (value 1 – 255)
d	decimal (32-bit)
u	unsigned decimal (32-bit)
x	hexadecimal lowercase
X	hexadecimal uppercase
o	octal (32-bits) (not supported on all of newLISP flavors)
f	floating point
e	scientific floating point
E	scientific floating point
g	general floating point

More info in Section 3.

1.8. List or string?

Probably in most cases, strings are more convenient for VisualNeo users than lists. In addition, many list functions can be applied to a string. However, to understand some useful newLISP functions completely, you have to know more about lists: see Section 3. More importantly, in Section 5 you’ll meet special lists that will benefit you.

Section 2. Quoting a list

As already said, a list as an argument in a function, needs to be quoted (= taken literally). We saw already an example:

(first ‘(1 2 3 4 5))
→ 1

Why is quoting necessary? Because newLISP interpretes the first element of a list as a function and the other elements as arguments of that function. Not quoting the list results in an error:

(first (1 2 3 4 5))
→ ERR: illegal parameter type in function first : 2

Remember that missing quotes are one of the most common causes of bugs in newLISP programs.

Section 3. A golden team of newLISP functions: ‘join’, ‘map’ and ‘apply’.

VisualNeo users can handle lists easily, thanks to the three functions ‘join’, ‘map’ and ‘apply’.

3.1 ‘join’ (and ‘explode’)

The function ‘join’ concatenates the given list of strings into a list containing one string.

(join ‘(“a” “b” “c”))
→ (“abc”)

(join ‘(“a” “b” “c”) “*”)
→ (“a*b*c”)

(join ‘(“a” “b” “c”) “*” true)
→ (“a*b*c*”)

The function ‘explode’ has an inverse operation of ‘join’:

(explode “newLISP”)
→ (“n” “e” “w” “L” “I” “S” “P”)

(join (explode “newLISP”))
→ “newLISP”

3.2 ‘map’

The function ‘map’ successively applies a function to its argument(s), returning all results in a list.

(map upper-case ‘(“a” “b” “c”))
→ (“A” “B” “C”)

(map + ‘(1 1 1) ‘(1 2 3))
→ (2 3 4)

3.3 ‘apply’

The function ‘apply’ applies a function to its argument(s) as a whole (and not as the function ‘map’ element by element).

(apply + ‘(1 2 3))
→ 6

which is the same as

(+ 1 2 3)
→ 6

Notice that ‘map’ returns a list, while ‘apply’ returns a string.

(map upper-case ‘(“abc”))
→ (“ABC”)

(apply upper-case ‘(“abc”))
→ “ABC”

Section 4. Learning by example

In this section I’ll present you some problems and its solutions. You’ll meet new functions, working together with ‘join’, ‘map’ and ‘apply’. A list or a string is the input and a string the output.

4.1. Remove duplicates in a list

(join (map string (unique ‘(1 1 2 2 2 2 2 2 2 3 2 4 4 3 2 4))))
→ “1234”

To remove duplicates in a string, you could write:

(join (unique (explode “1122222223244324”)))
→ “1234”

Notice that ‘unique’ performs faster with a sorted list.

(join (map string (unique (sort ‘(1 1 2 2 2 2 2 2 2 3 2 4 4 3 2 4)))))
→ “1234”

(join (unique (sort (explode “1122222223244324”))))
→ “1234”

4.2. Which characters in list 1 are not in list 2?

(join (map string (difference ‘(2 5 6 0 3 5 0 2) ‘(1 2 3 3 2 1))) “-“)
→ “5-6-0”

The same question can be applied to strings: which characters in string 1 are not in string 2?

(join (difference (explode “25603502”) (explode “123321”)) “-“)
→ “5-6-0”

4.3. Which characters do list 1 have in common with list 2?

(join (map string (intersect ‘(3 0 1 3 2 3 4 2 1) ‘(1 4 2 5)) ) “-“)
→ “2-4-1”

4.4. Ceil the numbers of a list and return a csv string.

(join (map string (map ceil ‘(1.01 2.53 3.64))) “-“)
→ “2-3-4”

In case you have a csv string with floats, you can do a simular operation:

(join (map string (map ceil (map float (parse “1.01;2.53;3.64” “;”))) ) “-“)
→ “2-3-4”

4.5. Combine the elements of two lists and return a two-element csv-string.

(join (map append ‘(“cats” “dogs” “birds”) (dup ” ” 3 true) ‘(“miaow” “bark” “tweet”)) “;”)
→ “cats miaow;dogs bark;birds tweet”

Do you understand what

(dup ” ” 3 true)

does? Well, it returns a list with three strings

(” ” ” ” ” “)

Notice that

(dup ” ” 3)

returns a string containing three spaces: ”   ”

In case you have a csv string as input, you could write:

(join (map append (parse “cats;dogs;birds” “;”) (dup ” ” 3 true) (parse “miaow;bark;tweet” “;”)) “;”)
→ “cats miaow;dogs bark;birds tweet”

Notice that the number 3 in the ‘dup’ function could be replaced by

(length ‘(“cats” “dogs” “birds”)) or (length (parse “cats;dogs;birds” “;”))

4.6. Reverse each word in this list: (To be or not to be!) and return a csv-string.

Recall that

(reverse ‘(To be or not to be!))
→ (be! to not or be To)

prints the original list backwards (which is also true if the argument is a string).

Now we have to use the function ‘map’ to apply the function ‘reverse’ to each word.

(join (map reverse (map string ‘(To be or not to be!))) ” “)
→ “oT eb ro ton ot !eb”

So the list is transformed into a list of strings. The same result is created by the function ‘parse’ applied to a string.

(join (map reverse (parse “To be or not to be!”)) ” “)
→ “oT eb ro ton ot !eb”

Section 5. Advanced list processing

The so called association lists, in fact a number of nested lists -consisting of two elements-, are a really nice extension of the list.

((1 “a”) (2 “b”) (3 “c”))

Essential to a association list is that the first element of the pair of elements is an index to the second. Two functions are relevant here: ‘assoc’ and ‘lookup’ extracts data of the association list.

5.1. ‘assoc’

(setq mc-data ‘((1 “a”) (2 “b”) (3 “c”)))
(assoc 3 mc-data)
→ (3 “c”)

5.2. ‘lookup’

(setq mc-data ‘((1 “a”) (2 “b”) (3 “c”)))
(lookup 3 mc-data)
→ “c”

BTW this example can also be made which multidimensional arrays (to be discussed in a next tutorial), which appear to be faster.

5.3. ‘assoc’ with ‘rest’

Of course, these functions can be combined with other list processing functions. Example:

(setq mc-data ‘((1 (a b)) (2 (b c)) (3 (c d))))
(rest (assoc 3 mc-data))
→ ((c d))

which is nearly equivalent to

(lookup 3 mc-data)
(c d)

To have the same result instead of ((c d)) -a list within a list-, apply ‘flat’:

(flat (rest (assoc 3 mc-data)))
→ (c d)

5.4. ‘find-all’

Use ‘find-all’ for searching a sublist in an association list that matches a pattern. ‘find-all’ accepts wildcards.

(setq math-score-semester-1-2019 ‘((John 7) (Tim 6) (Grace 8) (Jessica 7) ))

(find-all ‘(* 7) math-score-semester-1-2019)
→ ((John 7) (Jessica 7))

Or only the names by using the system variable $it:

(find-all ‘(* 7) math-score-semester-1-2019 (first $it))
→ (John Jessica)

Notice that ‘find-all’ has nice applications on simple lists: see www.newlisp.org/downloads/manual_frame.html

6. Bonus

I conclude this tutorial -just for fun- with two problems, where the golden team functions and regular expressions are applied to a string.

6.1. Split a string with a different number of the delimiter character ‘-‘ and return a csv string, delimited by only one ‘$’

(join (map title-case (parse “one-two–three—four” “-+” 0)) “$”)
→ “One$Two$Three$Four”

Notice that a regular expression that matches one or more occurrences of the hyphen-minus sign (-) is ‘-+’.

6.2. Split a string that contains different delimiter characters and return a csv string, delimited by only one ‘$’

(join (map upper-case (parse “hello, regular expression; why so tricky?” {(; *|, *| +)} 0)) “$”)
→ “HELLO$REGULAR$EXPRESSION$WHY$SO$TRICKY?”

Thank you for reading. Till next time!

Reinier Maliepaard

(last update: 18-08-2019)