In previous post I wrote how to invoke AppFabric management PowerShell commands from F# interactive. Now we successfully created named cache in AppFabric distributed cache and I want to use AppFabric cache API to store and retrieve data from this cache. You can structure data stored in named cache using cache regions (which roughly correspond to tables in relational database). Data could be stored in the region using string keys which work as in usual hashtable.

But first of all you need to find out how to refer to AppFabric API assemblies in F# project. MSDN documentation tells the following: “On 64-bit operating systems, the AppFabric directory will not be directly visible. To work around this problem, replace the System32 directory name with the name SysNative. This results in navigating to the C:\Windows\SysNative\AppFabric directory in this step”. So I added a reference to these assemblies in my F# script file:

F#, using GeSHi 1.0.8.8
#r @"C:\Windows\SysNative\AppFabric\Microsoft.ApplicationServer.Caching.Core.dll"
Parsed in 0.007 seconds at 11.99 KB/s

So far all looks OK, but when I selected this line and sent it to FSI window I’ve got this error message:

“error FS0084: Assembly reference 'C:\Windows\SysNative\AppFabric\Microsoft.ApplicationServer.Caching.Core.dll' was not found or is invalid”. I tried to search for this message but didn’t found any workaround. (It’s strange because this exact reference works perfectly when I added it to the project, but it still gives me an error in FSI window). To make it work in FSI I had to copy all needed AppFabric assemblies to the usual folder I created on my hard drive and refer to these copied assemblies from script file. It’s kind of ugly but it seems to work. So my references in script look like this:

F#, using GeSHi 1.0.8.8
#r @"C:\Projects\Velocity4\Microsoft.ApplicationServer.Caching.Core.dll"
#r @"C:\Projects\Velocity4\Microsoft.ApplicationServer.Caching.Client.dll"
#r @"C:\Projects\Velocity4\Microsoft.ApplicationServer.Caching.Management.dll"
#r @"C:\Projects\Velocity4\Microsoft.ApplicationServer.Caching.ManagementPS.dll"
#r @"C:\Projects\Velocity4\Microsoft.ApplicationServer.Caching.Configuration.dll"
 

Now I could use cache API.First of all we need to create DataCacheFactory object and DataCache client. (By the way MSDN has pretty good documentation on Windows Server AppFabric Caching Concepts and Architecture)

F#, using GeSHi 1.0.8.8
open System
open Microsoft.ApplicationServer.Caching
 
let createDataCacheFactory = 
    let config = DataCacheFactoryConfiguration()
    config.LocalCacheProperties <- DataCacheLocalCacheProperties() // use local cache in the cache client
    config.Servers <- [DataCacheServerEndpoint(Environment.MachineName, 22233)]
    new DataCacheFactory(config)
 
let createCacheClient cacheName = 
    let factory = createDataCacheFactory
    cacheName |> factory.GetCache
Parsed in 0.016 seconds at 29.76 KB/s

Now we can use DataCache object to manipulate data in our cache (I will not describe or implement complete cache API here but rather illustrate main concepts):

F#, using GeSHi 1.0.8.8
let putData (cache: DataCache) key value (region: string) =
    cache.Put(key, value, region)
 
let getData (cache: DataCache) key (region: string) =
    cache.Get(key, region)
 
let removeData (cache: DataCache) key (region: string) =
    cache.Remove(key, region)
 
let createRegion (cache: DataCache) regionName = 
    cache.CreateRegion regionName
 
let removeRegion (cache: DataCache) regionName =
    cache.RemoveRegion regionName
 
Parsed in 0.016 seconds at 27.77 KB/s

 

So we can now create some simple test data and test our F# API wrapper:

F#, using GeSHi 1.0.8.8
// Creating cache client for our named cache:
let cacheClient = createCacheClient "MyCache"
// Creating the region to store data
createRegion cacheClient "TestRegion"
// Creating simple test data
let testData = seq { 
    for i in 1 .. 10000 do
    yield (sprintf "key%d" i , sprintf "data%d" i) }
// Populate our cache region
testData |> Seq.iter (fun (fst, snd) -> putData cacheClient fst snd "TestRegion"|> ignore)
 
// Retrieve and print all data:
let getAllData = 
    for i in 1 .. 10000 do
    let key = sprintf "key%i" i
    let d = getData cacheClient key "TestRegion" 
    printfn "GotData: key:%s, value: %A" key d 
 
// We can play with different ways to get these data. 
 
open Microsoft.FSharp.Control;
 
let getAllDataMap testData = 
    testData |> Seq.map (fun (fst, _) -> getData cacheClient fst "TestRegion" )
 
let getAllDataParallel testData = 
    testData |> Seq.map (fun (fst, _) -> async { return getData cacheClient fst "TestRegion" })
            |> Async.Parallel
            |> Async.RunSynchronously 
 
// And remove them from cache finally:
for i in 1 .. 10000 do
let key = sprintf "key%i" i
removeData cacheClient key "TestRegion" |> ignore
 
Parsed in 0.028 seconds at 41.63 KB/s

 

So now we can use PowerShell  AppFabric management commands from F# and use cache API to put and get simple data. Next time I’m going to play with more advanced cache API (notifications etc.) and real database data.

Usually you can improve performance and scalability of distributed application using in-memory cache data. Windows Server AppFabric recently released by Microsoft includes distributed cache module formerly known under codename “Velocity”. It is a distributed in-memory application cache platform for developing scalable, available, and high-performance applications. AppFabric fuses memory across multiple computers to give a single unified cache view to applications. Applications can store any serializable CLR object without worrying about where the object gets stored. Scalability can be achieved by simply adding more computers on demand. The cache also allows for copies of data to be stored across the cluster, thus protecting data against failures. It runs as a service accessed over the network.

Installation and configuration went very smoothly, Microsoft documentation on setup and configuration is quite good.

I set up AppFabric on my laptop running 64-bit version of Windows 7 without any problems.

I planned to play with Cache API using F# interactive – a great tool available in Visual Studio 2010 allowing quickly prototype and test F# code.

Data stored in AppFabric cache are partitioned using named memory caches which could be created by PowerShell commands – PowerShell is the main tool for AppFabric administration tasks. Strangely enough PowerShell command is the only way to create new named cache on the server (there is no .NET API for this purpose) so I had to learn how to call PowerShell commands from F#. After some googling I found the great blog post by Stephen Kaufman “Calling PowerShell from .NET” which had exactly what I needed but he used C#. Both C# and F# are .NET languages so it shouldn’t be very hard to translate code from one to another. So I created a couple of functions to call PowerShell commands from F# and tried them in F# interactive.

 

F#, using GeSHi 1.0.8.8
// We need to refer PowerShell API assembly
#r @"C:\Program Files (x86)\Reference Assemblies\Microsoft\WindowsPowerShell\v1.0\System.Management.Automation.dll"
 
open System.Management.Automation
open System.Management.Automation.Runspaces
 
let invokePsCacheCommand cmdName (parNameVal: (string * obj) list) =
    let state = InitialSessionState.CreateDefault()
    // Import PowerShell AppFabric modules
    [|"DistributedCacheAdministration"; "DistributedCacheConfiguration"|]  |> state.ImportPSModule 
    state.ThrowOnRunspaceOpenError <- true
    let rs = RunspaceFactory.CreateRunspace(state)
    rs.Open()
    // Create PowerShell command pipeline
    let pipe = rs.CreatePipeline()
    pipe.Commands.Add(Command("Use-CacheCluster"))
    let cmd = new Command(cmdName)
    // Add command parameters as name-value pairs
    parNameVal |> List.iter (fun (name, value) -> cmd.Parameters.Add(CommandParameter(name, value)))
    pipe.Commands.Add(cmd)
    // And invoke commands
    pipe.Invoke()
 
let createCache cacheName = 
    [("Name", cacheName); 
     ("Expirable", box false)] |> invokePsCacheCommand "New-Cache"
 
let removeCache cacheName = 
    [("Name", cacheName)] |> invokePsCacheCommand "Remove-Cache"
 
let exportCacheConfig fileName = 
    [("File", fileName)] |> invokePsCacheCommand "Export-CacheClusterConfig"
 
Parsed in 0.026 seconds at 51.22 KB/s

 

Now I can try to create my first named cache from F# interactive:

 

createCache "MyCache"

 

I selected this string and hit Alt-Enter to execute it in F# Interactive window and immediately got a cryptic message: “The specified module 'DistributedCacheAdministration' was not loaded because no valid module file was found in any module directory.” After a short internet search I found this post by Ron Jacobs. It seems that I’ve got this error because I’m using 64-bit version of AppFabric but VisualStudio runs Fsi.exe as 32-bit process. Am I out of lack and cannot use F# interactive to run PowerShell commands? No. It seems that in quickly expanding F# community one can find the answer to almost any problem:) Here it is: “Making the FSharp interpreter (fsi) run in 64bit!” I made all woodoo magic procedures described in this article and finally got sweet picture in power shell command window:

 

 

In the next part I’ll describe how to use cache API to store and retrieve data using F#.

 

IT University of Copenhagen uses F# as presentation language in a course “Programming Language Concepts for Software Developers”

Link to the course material (textbook, presentations and exercises):

http://www.itu.dk/courses/BPRD/E2009/

It was pretty cool. Steffen Forkmann spoke from Germany for groups here in Boston and New Orleans (at TechEd 2010). Looks like F# is almost ideal for creating DSLs. Discovered new F# feature which allows create functions like this: 

  let ``function name with spaces и всяческим уникодом`` num =

      printfn "Output from the function %d" num  

Just wrap the arbitrary string by double ticks `` – and it will be perfectly legal F# identifier!

Fsi output: val ( function name with spaces и всяческим уникодом ) : int -> unit

> ``function name with spaces и всяческим уникодом`` 45;; Output from the function 45 val it : unit = () >

Vladimir Matveev implemented yet another variant of high-performant F# events using Linq expression trees.

Lambdas in Java: An In-Depth Analysis

The lambda draft is progressing, although not in the originally proposed timeframe of the JDK 7 release plan. With the recent addition of defender methods as well, it is unlikely that a full implementation will be ready for this Summer. Whether JDK 7 will ship without lambdas (or without a retro-fitted Collections API) or whether JDK 7 will be delayed to accommodate the lambda extensions remains to be seen.

New England F# User Group

Our next meeting will take place on Monday, June 7th from 6:30pm to 8:30pm!
All interested are welcome to join us. 

Speaker: Steffen Forkmann, author of FAKE and NaturalSpec 

Steffen is the author of the two open source F# projects "FAKE - F# Make" (http://bitbucket.org/forki/fake/wiki/Home) and "NaturalSpec" (http://bitbucket.org/forki/naturalspec/wiki/Home).  In this talk, he shows these tools in action, and talks a bit about some internal implementation details. In particular he shows why he used F# and how specific language constructs help to simplify the usage.

Steffen Forkmann is currently a developer for msu solutions GmbH in Hamburg (Germany), where he helps to develop a large accounting system based on Microsoft Dynamics NAV. In addition to his day job, he has a deep interest in functional programming and in particular in F#. You can find him on his blog at http://www.navision-blog.de.

Microsoft Research New England
First Floor Conference Center
One Memorial Drive
Cambridge, MA 02142

DelegateEvents use DynamicInvoke under the hood and so can be slow when trigged frequently.  As we have an existing model which requires an event interface, this was was causing our F# implementation to be a bit slower than the C# we were comparing it to.  Thankfully, F# allows you to implement your own eventing via IDelegateEvent.  With a standard Invoke version of DelegateEvent provided by James Margetson, our implementation in F# is running at equivalent speed to C#.

1) Naive implementation:

 

1. let rec fibnaive n =
2.     if n < 2 then 1I
3.     else
4.         fibnaive (n - 1) + fibnaive (n - 2)
5.  

 

Horrible performance. fibnaive 40 took 16.6 sec on my laptop
 

2) Matrix algorithm:

It’s handy that F# have matrix type in Microsoft.FSharp.Math namespace.

The type matrix uses float numbers. Let’s see how we could calculate Fibonacci’s:

F#, using GeSHi 1.0.8.6
open Microsoft.FSharp.Math
let matrixd = matrix [ [ 1.; 1. ];[ 1.; 0. ] ]
let rec fibmatd n =
    if n < 2 then matrixd
    else matrixd * fibmatd(n - 1)
let fib n =
    if n < 2 then 1I
    else
let m = fibmatd n
System.Numerics.BigInteger(m.[1, 0])
 
Parsed in 0.013 seconds at 21.61 KB/s

Result of fib 1000:

Real: 00:00:00.002, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : System.Numerics.BigInteger =
4346655768693742750699615555111373698901493060675
3135940421893445574924103291704556590724031629457
1302242425620098031386057959852750558837247446456
7001785294423082188544814790061823421628580750638
3791353495552

It’s pretty fast but it has a problem: if I remember correctly Fibonacci of 1000, it calculates only first 15 digits right.

3) Generic matrix:

But this namespace has generic version of matrix type – we can use any arbitrary type to construct our matrix. We would use BigInteger to get a correct results.
 

F#, using GeSHi 1.0.8.6
open Microsoft.FSharp.Math
let anymatrix = Matrix.Generic.ofList
let fibceed = anymatrix [ [ 1I; 1I ];[ 1I; 0I ] ]
let rec fibmat n =
    if n < 2 then fibceed
    else fibceed * fibmat(n - 1)
let fib n =
    if n < 2 then 1I
    else
let m = fibmat n
    m.[0, 1]
 
Parsed in 0.012 seconds at 23.16 KB/s

Now the result of fib 1000:
Real: 00:00:00.006, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : System.Numerics.BigInteger =
4346655768693745643568852767504062580256466051737
1780402481729089536555417949051890403879840079255
1692959225930803226347752096896232398733224711616
4299644090653318793829896964992851600370447613779
5166849228875

It’s slightly slower but it calculates Fibonacci of 1000 correctly.

4) Another algorithm:

F#, using GeSHi 1.0.8.6
let rec fibb (n1: bigint)  (n2: bigint) c =
    if c = 1 then
n2
    else
fibb n2 (n1+n2) (c-1)
 
let fibfast num =
    (fibb 1I 1I (num - 1))
 
Parsed in 0.009 seconds at 17.89 KB/s

It’s results are so far fastest I could get:

Real: 00:00:00.001, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0

val it : bigint =
4346655768693745643568852767504062580256466051737
1780402481729089536555417949051890403879840079255
1692959225930803226347752096896232398733224711616
4299644090653318793829896964992851600370447613779
5166849228875

And it’s tail recursive so we can play with it up to OutOfMemory exception for very very big integers. I’m always wanted to know what would be Fibonacci of 1000000. And now I can get it in slightly more than 2 min:

Real: 00:02:06.986, CPU: 00:02:10.229, GC gen0: 18414, gen1: 2523, gen2: 395
I don’t put it here because it consists of 208993 digits, but it starts with:

19532821287077577316320149475962563324435…
 

5) And finally using Seq type:

F#, using GeSHi 1.0.8.6
let fibonacci =
Seq.unfold
     (fun (current, next) -> Some(current, (next, current + next)))
    (0I, 1I)
 
let term n= fibonacci |> Seq.take n
let fib 1000 = Seq.nth 1000 (term 1001)
 
Parsed in 0.012 seconds at 17.07 KB/s

It’s also pretty fast. Here are results of calculating 1000 Fibonacci numbers:
Real: 00:00:00.001, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0

If anyone have another interesting algorithm to share – welcome to comments!

Here I want to test how to post code snippets to this community. As an example I put here code from “F# Expert” book. I'm using Live Writer with GeSHi plugin for syntax highlighting.

The sample shows how using higher order functions and Linq we can easily and elegantly create XML documents.

F#, using GeSHi 1.0.8.6
open System.Xml.Linq
 
let xml =
"<contacts>
        <contact>
            <name>John Smith</name>
            <phone type=\"home\">+1 626-123-4321</phone>
        </contact>
     </contacts>"
let doc1 = XDocument.Parse xml
 
let xname n = XName.op_Implicit(n)
let xdoc (el:seq<XElement>) = new XDocument(Array.map box (Array.ofSeq el))
let xelem s el = new XElement(xname s, box el)
let xatt a b = new XAttribute(xname a, b) |> box
let xstr s = box s
 
let doc =
xdoc
    [xelem "contacts"
        [xelem "contact"
            [(xelem "name" (xstr "John Smith"))
             (xelem "phone"
                 [xatt "type" "home"
                  xstr "+1 626-123-4321"])
            ]
         ]
    ]
Parsed in 0.022 seconds at 34.35 KB/s