List Utilities Module Users' Manual

Purpose

This document discusses the list_utils module and the predicates exported from this module. Generally, lists can be considered to be the fundamental, essential, non-atomic data type, and, as such, a body of standard functionality has grown to be canon around this data type. This module implements the most-used functionality from that canon and presents it to the Prolog user as one protocol.

This module follows the pattern of list-manipulation taken by the functional programming community, which is to use "functionals" (constructed functions passed in as arguments) to transform the list from its original state to another (possibly resulting in an object that is not a list) state. Given this, it is necessary to understand how to construct a functional that does the work necessary to transform the list into the state desired: both the combinators and the lambda modules provide protocols to construct functionals.

One must also consider the non-functional approach to exhaustive list-processing: this module for the most part takes the exhaustive approach as the way to perform list-processing, but there are other approaches that better fits the situation at hand, particularly for Prolog. In cases where non-determinism or choice is more important than a comprehesive scan, then operators from the syntax module are more appropriate, particularly ones that select a member or use (reverse- or bi-directional-) implication to narrow a search. This module follows the functional, comprehesive, approach to list-processing, so use this module when the entire list must be transformed; look elsewhere (such as transforming the list into a set or bag and then operating on the new data type) when logic or selection is desireable.

Functionality Not Implemented

The following set of canonical functions have not been implemented in this module. When the need arises for any of these functions, they will be added as necessary. These functions are part of the canon, so their functionality is defined in a variety of sources.1

zip(+ListA, +ListB, ?List)
List is the result of alternating the elements of ListA and ListB, starting with the first element of ListA (followed by the first element of ListB, then the second element of listA, then the second element of ListB, etc.). When there are no more elements remaining in one of the lists, the remainder of the nonempty list is appended.
foldr(+Pred, +List, +Accum, ?Ans)
The same functionality of foldl/4, except that the List is folded starting from the last element (the right of the list) instead of the first.
replicate(+Integer, +Seed, ?List)
List is Seed repeated Integer times. unfold/5 is a generalization of this predicate.
 
The following predicates assume normal order ("lazy") evaluation as they give infinite results. We would need to build in a facility for Prolog to evalute the results incrementally ("lazily") in order to be able to implement these predicates.
 
iterate(+Pred, +Seed, -List)
Gives an infinite List from repeated successive application of Pred to Seed.
List = [Seed, Pred(Seed), Pred(Pred(Seed)), ...].
repeat(+Seed, -List)
Gives an infinite List from repeating Seed infinitely. The same as List = [Seed|List].
cycle(+List, ?Circle)
Takes a (possibly cyclic (infinitely repeating)) list and makes it cyclic: Circle = List ++ Circle.

:- module(list_utils)

Dependences

:- use_module(library(lambda), [apply/3, test/2]).

:- use_module(library(peano), [peano/2]).

:- use_module(library(combinators), [phi_equivalent/2, compose/3]).

:- use_module(library(syntax), [try/3]).

Exported Predicates

The list_utils module exports the following predicates:

Exported Operators

This module exports no operators.

map_reversed/4


Synopsis: map_reversed(+Functor, +List, +DCGIn, -DCGOut)
Arguments:  

Functor

<term> A phi-equivalent term (which may be a lambda/2-term, a combinator, or a phi/3-term)
List <list> A proper list whose elements are to be transformed by Functor
DCGIn <list> The proper list onto which the transformed elements from List are to be prepended
DCGOut <list> The proper list resulting from concatenating the reverse of List tranformed by Functor and DCGIn
Description: 

Recursively applies Functor to the elements of List, prepending the result onto DCGIn. In short, map_reversed/4 gives a reversed, tranformed, list.

Backtracking: 

Standard. N.B.: one cannot use the trick of having DCGIn free and DCGOut bound to the initial state in the hope of obtaining a transformed result in DCGIn in the element ordering of List: this predicate fails for that case. Use map/3 if element ordering should be preserved.

Example: 

Here we demonstrate accumulating a set of similarly operating rules (specifically, a set of rules that take no parameters) onto a set of rules with parameters already assigned for testing by a unit test framework:

  map_reversed(lambda(RuleID, rule(RuleID, [], 180)),
	       ['1a', '1b', '1c', '9a', '9b', '9c', '9d', '17a',
	        '28a', '28b', '28c', '33a', '37a', '51a'],
	       [rule('24a', Param24ab, 42), rule('24b', Param24ab, 42),
	        rule('24c', Param24c, 42), rule('31a', Param31a, 180),
		rule('34a', Param34a, 180), rule('34b', Param34b, 180),
		rule('34c', Param34c, 180),
		rule('49a', Param49a, 180), rule('51b', Param51b, 180)],
               NoScoreConstructs)
See also: 

map/3, reverse/2

map/3


Synopsis: map(+Functor, +ListIn, ?ListOut)
Arguments:  

Functor

<term> A phi-equivalent term (which may be a lambda/2-term, a combinator, or a phi/3-term)
ListIn <list> The proper list to be transformed by Functor.
ListOut <list> The proper list result
Description: 

Recursively applies Functor to the elements of ListIn, resulting in ListOut containing the tranformed elements with the same ordering as ListIn. The research from the functional programming language community has shown that all (list-)transformation functions are derivatives of foldl/4, but this predicate is so commonly used that the justification for its independent definition is self-evident.

Backtracking: 

If ListOut is free, deterministic; otherwise, semideterministic based on the unification of the map/3 result and the input, bound, ListOut.

Examples: 

In this example, we have a party (HiLo) that we pair with a each party in an input list:

map(lambda(Party, party([HiLo, Party])), Parties, Pairs)

This next example continues from the result of the map_reversed/4 example: we transform the rule/3 terms into a nominal/1 term that instructs the unit test framework that the correct result is no rule finding based on the input data:

map(lambda(rule(ID, Params, Period),
	   nominal(no_rule(ID, reason(zero_score_rule),
                           InnocuousData, Duration, Params,
			   Period, InitEnv))),
    NoScoreConstructs, NoScoreNoRules)

Incidentally, we see from this example that the input variable of a lambda-term can be destructured just as any other Prolog unification.

See also: 

map_reversed/4, foldl/4

filter/4


Synopsis: filter(+List, +Match, ?Trues, ?Fails)
Arguments:  

List

<list> The input list of elements to be divided into the output lists of Trues and Fails.

Match

<term> Match is a binary type: it is either a phi-equivalent term (which may be a lambda/2-term, a combinator, a psi/2-term, or a phi/3-term) or a unification term in the shape of the elements of List used to discriminate elements in List.
Trues <list> The proper list composed of the elements of List that prove Match.
Fails <list> The proper list of elements of List that fail to prove Match.
Description: 

Converts List into two lists, one that has elements that match Match and the other with the elements that do not. filter/4 can be considered a generalization of map/3 in that filter/4 retains the elements that do not prove Match as well as the ones that do.

Backtracking: 

If Trues and Fails are free, deterministic; otherwise, semideterministic based on the unifications of the filter/4 results and the input, bound, output lists.

Examples: 

In this example, we create Match that is a unification term discriminator, filtering a set of rules into ones with at least one parameter and ones with none:

filter(Rules, rule(_, _, _, parameters([_|_]), _), ParamsRules, NoParamRules)

This next example uses a psi/2-term to filter a list based on a predicate:

X := 1..10, filter(X, psi(Y, Y < 5), Yes, No)

The result of the above call is that Yes = [1,2,3,4] and No = [5,6,7,8,9,10]

See also: 

map/3

foldl/4


Synopsis: foldl(+List, +BinaryFunctor, +Seed, ?Ans)
Arguments:  

List

<list> The input list of elements

BinaryFunctor

<term>

A composite phi-equivalent term of the form: phi(Head, phi(Accum, NewAccum, Phi1), Phi0),

where (in the order of execution):
  Head <term> the element at the current head of List
  Phi0 <callable> the goal to execute once the phi-term receives Head; usually the goal true/0 is appropriate.
  Accum <term> The current value of Seed
  Phi1 <callable> the goal to execute once the phi-term receives Accum; this goal uses Accum and Head and puts the result of the goal into NewAccum.
  NewAccum <term> The (final) result of the execution of this phi-term.
Seed <term> The starting value of the fold operation; recursively updated by BinaryFunctor.
Ans <term> The result of accummulated application of BinaryFunctor to the elements of List and to Seed.
Description: 

This is the fundamental (program) transformation function, given to us by research into functional programming.2 This predicate takes the elements of a list and an initial value and recursively applies BinaryFunctor to the two obtaining a final result (which, only in the special case of the secondary function of the composition being concatenation does a list result, otherwise the result is a non-list term). foldl is also known as reduce;3 although not strictly correct, this name does capture the majority of this predicate's applicability.

foldl/4 captures the mechanics of (terminating, or eager, or applicative) recursion4 in predicate form. So, using this predicate in conjuction with an appropriate set of elements in List (such as a list of combinators) and an appropriate BinaryFunctor will allow one to build functions on-the-fly. Given the above, and given that most functional programs can be treated as data (i.e. Lisp programs are lists; and lists are Lisp's data structure), it is possible to feed programs to foldl to obtain new programs; therefore it is associated with being a '"program" transformation' function.

Backtracking: 

depends on the modality of Ans and BinaryFunctor (e.g. if Ans is free and BinaryFunctor is deterministic, then this instance of foldl is deterministic as well).

Examples: 

Here is a simple definition of sum_list/2:

foldl(List, "+", 0, Ans)
so, X := 1..10, foldl(X, "+", 0, 55) holds.

Here is a program that multiplies ratios, given a fixed denominator:

bayes_preference(List, Denominator, Result) :-
  compose("`c/", Denominator, Ratio),
  compose("`b*", Ratio, Multiplier),
  foldl(List, Multiplier, 1.0, Result).

So, for example, let's say a Bayesian system has collect counts over seven attribute for 595 occurrences of a class, if the counts of a new input matched the following:

Attributes = [412,485,564,363,75,15,420]

then bayes_preference(Attributes, 595.0, Result) would give the value that shows the preference toward that class (Result is approximately 7.3e-4).

See also: 

The dual of foldl/4 is unfold/5

unfold/5


Synopsis: unfold(+StopPred, +CurrPred, +NextPred, +Seed, ?Ans)
Arguments:  

StopPred

<term> a psi-equivalent term that if true when composed with the current value of Seed, stops the process of induction, giving the accumulated value of Ans as a result.

CurrPred

<term> a phi-equivalent term that gives the current value to be accumulated onto Ans when composed with Seed

NextPred

<term> a phi-equivalent term that gives the next value of Seed on the next inductive call to this predicate.
Seed <term> The starting value of the unfold operation; recursively updated by NextPred.
Ans <list> The list resulting from this operation.
Description: 

This predicate serves the need of obtaining a list from a starting value and a termination test. In essence, this is the mathematical, or functional, definition of a series.5 This predicate implicitly builds a list instead of requiring the programmer to explicitly enumerate the elements (by defining the list by hand or by creating and calling a recursive predicate).

Backtracking: 

Nondeterministic; choice-points are left at each inductive call in the construction of Ans.

Examples: 

The set_utils module defines the arithmetic sequence operator thusly:

'..'(Count, X, Y) :- unfold(psi(A, A > Y), i, "`+1", X, Count).

It may be a fruitful line of research, given monadic tranformations6 to allow unfold/5 to work with monads for CurrPred and NextPred, as monads facilitate dynamic programming, so that, for example, sequences that depend on the values of earlier elements (moreso than simply the most recent element) may be defined. Both the Fibonacci and prime sequences could be defined with unfold/5 using monads, whereas now, the simplest approach for defining these sequences is to use explicit (recursive) predicates.

See also: 

'..'/2; The dual of unfold/5 is foldl/4


Endnotes

1

For example, The appendix with the prelude definition of Haskell in the appendix of [Dav1992], or the Haskell manual itself (in the Standard Prelude section), obtainable from http://haskell.org/onlinereport/standard-prelude.html, or, from the source we use, the Mercury library reference manual, particularly the list module section, available from http://cs.mu.oz.au/research/mercury.

2

Because foldl is a function, not a predicate the standard signature for it is...

foldl(BinaryFunctor, Seed, List) = Ans

... in the functional programming community. We have a different signature for this module. Deal with it.

3

The programming language Dylan names this function reduce, not foldl. The Gwydion Dylan Maintainers are the current and active developers of the Dylan programming language, their entry web-page is http://www.gwydiondylan.org.

4

I placed all the caveats to describe foldl's recursion, because it is trivial to capture the definition of normal-order, or lazy, recursion using combinators. In fact, there exists a recursion-defining combinator which is known in the literature as the 'y' or 'θ' combinator or the 'sage bird'. The "problem" with the combinator approach is that it defines recursion generally, with the result of it defining it infinitely, and allowing the user of the recursion to take the first 'N' iterations for the specific task. As Prolog, by default, evaluates arguments eagerly, a call using that combinator would never return. To be able to use combinatory recursion, a normal-order evaluation system must be developed for Prolog first. This task currently falls outside of the scope of this module.

5

Of course, series, and their construction, has been the study of mathematical philosophers since ancient times. The serious study of series have entered into the computer science field with the adoption of the series type into Common Lisp (see, e.g., [Wat89], or the Series appendix of [Ste84]), further refined by the list construction syntax in the Haskell programming language (see, particularly, §3.10 and §3.11 of [Jon2002]).

6

A good collection of essays on monads can be found at http://haskell.cs.yal.edu/bookshelf/#monads. Unfortunately, the link to the excellent, practical, introductory article: "What the hell are Monads?" no longer functions.


Works Consulted

[Dav1992] An Introduction to Functional Programming Using Haskell, A. J. T. Davie, Cambridge Computer Science Texts, UK, 1992.
[Jon2002] Haskell 98 Language and Libraries, the Revised Report, Simon Peyton Jones, ed., et al, http://www.haskell.org/onlinereport, December 2002.
[Ste1984] Common Lisp: the Language, 2nd ed., Guy L. Steel, Jr., Digital Press, 1984.
[Wat1989] Richard C. Waters, "Optimization of Series Expression: Part II: Overview of the Theory and Implementation", A.I. Memo No. 1083, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, December 1989.

Author:Douglas M. Auclair
Created:October 14, 2005
Last Modified:October 17, 2005