In the spelling corrector article we read a whole file in memory and hold it’s data in a std::string object, and then we use a std::map to hold ranges of this big string. Those ranges are created by making copies of the string. While this aproach is satisfatory, it can be greatly improved if we can afford holding the file data while std::map is in the scope.

That’s because we may hold only information about the string range in the map, not the whole copied string, this will greatly improve spelling corrector loading time:

template<class CharT, class Traits = std::char_traits<CharT> >
class basic_const_string_range
{
public:

  typedef CharT value_type;
  typedef Traits traits_type;
  typedef const value_type* const_iterator;
  typedef std::string::size_type size_type;

  size_type size() const { return m_size; }

  basic_const_string_range(const_iterator begin, size_type size)
    : m_begin(begin)
    , m_size(size)
  {
  }

  bool operator<(const basic_const_string_range<CharT>& other) const
  {
    return traits_type::compare(begin(), other.begin(),
      other.size() > size() ? size() : other.size()) < 0;
  }

  const_iterator begin() const { return m_begin;          }
  const_iterator end()   const { return m_begin + size(); }

private:

  const_iterator m_begin;
  size_type m_size;
};

template<class CharT>
basic_const_string_range<CharT> make_range(const CharT* begin, std::string::size_type count)
{
  return basic_const_string_range<CharT>(begin, count);
}

typedef basic_const_string_range<char>    const_string_range;
typedef basic_const_string_range<wchar_t> const_wstring_range;

int main()
{
  std::string str = "apple banana orange ";
  std::map<const_string_range, int> data;

  std::string::size_type first_non_space = 0;
  std::string::size_type last_space      = 0;
  while ((first_non_space = str.find_first_not_of(' ', first_non_space)) != std::string::npos &&
         (last_space      = str.find             (' ', first_non_space)) != std::string::npos)
  {
    data[make_range(&str[first_non_space], last_space - first_non_space)] = first_non_space;
    first_non_space = last_space + 1;
  }
}

This class can be adapted to a TR1 unorderep_map providing the correct hash<> overload. The map will continue valid as long as the string object remains in the memory. Yet it has another use, consider for example this input:

int lookup(const char* data)
{
  return values[data];
}

In a normal std::map<std::string, int> when we called operator[](const std::string&) the variable data would call the std::string constructor and copy all it’s data to the string internal buffer. Using the basic_const_string_range approach we can save that copy using the const char* variable directly.

I did a filter_iterator for C++ STL containers which is able to filter container elements based on a predicate, wich I have showed to my friend Thiago Adams. Then he pointed me a CUJ (C/C++ Users Journal) article where Maciej Sobczak introduced the Container View Concept. The main idea behind it is that it is able to move elements without really copying them and being able to integrate multiple container elements in just one view.

The View contains a collection of pointers to other elements. So the basic interface of the view would look like:

const std::string names1[] = { "felipe", "thiago" };
const std::string names2[] = { "joao", "andre", "bruno" };

view<const std::string> names_view(&names1[0], names1 + 2);
names_view.assign(&names2[0], names2 + 3);

std::sort(names_view.begin(), names_view.end());
std::copy(names_view.begin(), names_view.end(), std::ostream_iterator<const std::string>(std::cout, "\n"));

So, the idea is that the view<> object will be sorted (his pointers will point to the elements in the arrays in a sorted way) while the arrays itself will remain untouched.

In Maciej’s post it’s view was returning the pointer to the elements directly, so when he called std::sort he had to use an adapter in the predicate function to dereference the pointer to the elements. Actually, what we really want in this case is to have a container to store references to objects, but everyone knows that’s impossible… right?

Not really, TR1 has arrived with a class called reference_wrapper<T> wich basically defines a copyable reference, so:

  int data[] = { 51,9,0,5,4 };
  std::vector<std::tr1::reference_wrapper<int> > view;

  for (int i = 0; i < 5; ++i)
    view.push_back(std::tr1::ref(data[i]));

  std::sort(view.begin(), view.end());
  std::copy(view.begin(), view.end(), std::ostream_iterator<int>(std::cout, "-"));

Ok, that’s a pretty nice line reduction compared to the article , and now we don’t need the special predicate in sort for it to work. But problems might still arise:

1) It is annoying to have to use a loop to push_back elements in our view.

2) The reference_wrapper<T> doesn’t define an operator= for us to be able to assign a new value to the reference. This can create problems with for example the std::replace algorithm defined in <algorithm>.

With these requirements we can create a class that better adresses our needs:

template<class T>
class view_reference
{
  T* value_;
public:
  view_reference(T& value)
    : value_(&value)
  {
  }

  operator T&()
  {
    return *value_;
  }

  view_reference<T> operator=(const T& value)
  {
    *value_ = value;
    return *this;
  }
};

void main()
{
  int data[] = { 51,9,0,5,4 };
  std::vector<view_reference<int> > view;

  std::copy(&data[0], data + 5, std::back_inserter(view));
  std::sort(view.begin(), view.end());
  std::replace(view.begin(), view.end(), 51, 3000);

  std::copy(view.begin(), view.end(), std::ostream_iterator<int>(std::cout, "-"));
}

So now we got a nice view container, hope you enjoy it .

Greetings, i wrote a C++ version of the Norvig’s spelling corrector. All theory and other language implementations can be found in his website. Here’s my reduced version with less number of lines, that scored 66 lines of code (Implementation in the same source that the declaration): SpellCorrector in C++. Or just take a look at the complete version:

If you want to use the Spelling Corrector in your project feel free, but just send me an email to let me know about it.

SpellCorrector.h

/*
 * SpellCorrector.h
 *
 * Copyright  (C)  2007  Felipe Farinon <felipe.farinon@gmail.com>
 *
 * Version: 1.4
 * Author: Felipe Farinon <felipe.farinon@gmail.com>
 * Maintainer: Felipe Farinon <felipe.farinon@gmail.com>
 * URL: http://scarvenger.wordpress.com/
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 *
 * Commentary:
 *
 * See http://scarvenger.wordpress.com/.
 *
 * Code:
 */

#ifndef _SPELLCORRECTOR_H_
#define _SPELLCORRECTOR_H_

#include <vector>
#include
<tr1/unordered_map>

class SpellCorrector
{
private:
	typedef std::vector<std::string> Vector;
	typedef std::tr1::unordered_map<std::string, int> Dictionary;

	Dictionary dictionary;

	void edits(const std::string& word, Vector& result);
	void known(Vector& results, Dictionary& candidates);
public:
	void load(const std::string& filename);
	std::string correct(const std::string& word);
};

#endif

SpellCorrector.cpp

/*
 * SpellCorrector.cpp
 *
 * Copyright  (C)  2007  Felipe Farinon <felipe.farinon@gmail.com>
 *
 * Version: 1.4
 * Author: Felipe Farinon <felipe.farinon@gmail.com>
 * Maintainer: Felipe Farinon <felipe.farinon@gmail.com>
 * URL: http://scarvenger.wordpress.com/
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 *
 * Commentary:
 *
 * See http://scarvenger.wordpress.com/.
 *
 * Code:
 */

#include <fstream>
#include <string>
#include <algorithm>
#include <iostream>

#include "SpellCorrector.h"

using namespace std;

bool sortBySecond(const pair<std::string, int>& left, const pair<std::string, int>& right)
{
	return left.second < right.second;
}

char filterNonAlphabetic(char& letter)
{
	if (isalpha(letter))
		return tolower(letter);
	return '-';
}

void SpellCorrector::load(const std::string& filename)
{
	ifstream file(filename.c_str(), ios_base::binary | ios_base::in);

	file.seekg(0, ios_base::end);
	int length = file.tellg();
	file.seekg(0, ios_base::beg);

	string line(length, '0');

	file.read(&line[0], length);

	transform(line.begin(), line.end(), line.begin(), filterNonAlphabetic);

	string::size_type begin = 0;
	string::size_type end   = line.size();

	for (string::size_type i = 0;;)
	{
		while (line[++i] == '-' && i < end); //find first '-' character

		if (i >= end) { break; }

		begin = i;

		while (line[++i] != '-' && i < end); //find first not of '-' character

		dictionary[line.substr(begin, i - begin)]++;
	}
}

string SpellCorrector::correct(const std::string& word)
{
	Vector result;
	Dictionary candidates;

	if (dictionary.find(word) != dictionary.end()) { return word; }

	edits(word, result);
	known(result, candidates);

	if (candidates.size() > 0) { return max_element(candidates.begin(), candidates.end(), sortBySecond)->first; }

	for (unsigned int i = 0;i < result.size();i++)
	{
		Vector subResult;

		edits(result[i], subResult);
		known(subResult, candidates);
	}

	if (candidates.size() > 0) { return max_element(candidates.begin(), candidates.end(), sortBySecond)->first; }

	return "";
}

void SpellCorrector::known(Vector& results, Dictionary& candidates)
{
	Dictionary::iterator end = dictionary.end();

	for (unsigned int i = 0;i < results.size();i++)
	{
		Dictionary::iterator value = dictionary.find(results[i]);

		if (value != end) candidates[value->first] = value->second;
	}
}

void SpellCorrector::edits(const std::string& word, Vector& result)
{
	for (string::size_type i = 0;i < word.size();    i++) result.push_back(word.substr(0, i)             + word.substr(i + 1)); //deletions
	for (string::size_type i = 0;i < word.size() - 1;i++) result.push_back(word.substr(0, i) + word[i+1] + word.substr(i + 2)); //transposition

	for (char j = 'a';j <= 'z';++j)
	{
		for (string::size_type i = 0;i < word.size();    i++) result.push_back(word.substr(0, i) + j + word.substr(i + 1)); //alterations
		for (string::size_type i = 0;i < word.size() + 1;i++) result.push_back(word.substr(0, i) + j + word.substr(i)     ); //insertion
	}
}

I am working in some optimizations yet but it should be ready to use. Using the corrector is pretty easy, just download the words database, and see the example:

#include "SpellCorrector.h"

using namespace std;

int main()
{
    SpellCorrector corrector;

    corrector.load("big.txt");

    string request;
    while (request != "quit")
    {
        cout << "Type one word\n";
        cin >> request;

        string correct(corrector.correct(request));

        if (correct != "")
            cout << "Did you mean: " << correct << "?\n\n\n";
        else
            cout << "No corrections avaiable\n\n\n";
    }

    return 0;
}