RAM drives and OS X

[Dave2002]

I just found a few articles about RAM drives, and here is one about OS X - http://www.tekrevue.com/tip/how-to-c...k-in-mac-os-x/

I hadn't realised that even SSDs don't really compare for speed with RAM drives - though the warnings about data volatility should be noted.

Also amazed that the comparison in the article was with a 240 Gbyte SSD - some people obviously take memory issues to very expensive lengths.
However, the max data size in the graph is around 256 Mbytes, and some files could be bigger than that. Maybe that's only a throughput value for the speed tests though, so not a limit on file size.

This article certainly suggests that anyone contemplating an upgrade rather than a machine replacement should seriously consider maxing out the memory.

However, getting the balance right between main memory and memory used for any RAM drives could be tricky. 16 Gbytes memory might just not be enough for some applications, in which case it might be better just not to bother. An example might be making a DVD in which the final output data file might be around 4 Gbytes, and intermediate files could be similar. That might leave about 8 Gbytes of working memory in the machine. OTOH if the whole 16 Gbytes is left to the OS, although speed could be a lot lower, the in built memory management system could sort things out to get the job done, and there'd be lower risk of a crash or other problem wiping everything out.

If it is possible to work on video material in sections, to avoid the max sizes, then RAM drives could certainly speed things up.

[Anastasius]

This article is two years old I think you will find that Apple have effectively implemented that with their memory management system in later versions of OSX from Mavericks onwards.

[Dave2002]

This article is two years old I think you will find that Apple have effectively implemented that with their memory management system in later versions of OSX from Mavericks onwards.

WIthout having an inside track on Apple's design I can't say at present. If I upgrade the memory on my iMac in the near future (this coming week perhaps) I may be able to do some form of black box testing which might give clues, though I doubt that anything would be very conclusive. One problem would be that Apple's memory management system would not necessarily have the best view of the applications - though may be able to get a good "view" of what's going on. Are there any articles which give clues as to what Apple have done? It looks as though the memory management system does some form of on the fly compression and decompression when memory gets tight to try to avoid reading or writing any data from/to disk. With a much larger physical memory installed there'd be no need to do that most of the time.

It might also be possible to keep most of the active program pages in real memory most of the time - there'd be no need to write them back to file or disk, and similarly for read only data pages. Clever heuristics might be required to optimise the behaviour when reading and writing large files. Maybe Apple have got all this sussed out, but they would say they had wouldn't they?

[Anastasius]

There is this article http://www.switchingtomac.com/tutori...s-x-mavericks/ but I doubt it will satisfy you as you do seem set in your view of how memory should be used/memory display in Activity Monitor.

It works. It works well as far as I can see and that is good enough for me.

Of more import, IMO, is that Google 'phones home' very frequently to check for updates on all the Google programs that you might have installed on your Mac. Whether those programs are running or not. And without so much as a by-your-leave. There is a fair bit on the web about this...a process called Keystone Agent..and some terminal commands that are supposed to tell it to stop. However, I suspect that Google have, since these tips came out, fiddled around with Keystone Agent as none of these worked in my case. I eventually found a command to nuke it. And that has certainly stopped it dead in its tracks. I just need to make sure that I manually check Chrome for updates on the rare occasion that I use it.

The other thing that happened recently here (and apologies for going OT) was an errant IMAP client in Apple Mail that continuously kept requesting to sync with the server and as a result consumed nearly 500MB of IMAP bandwidth in two days !!

Console logs are very revealing...it's where I found Keystone Agent.

[Anastasius]

And here's another thing that I think you will find interesting especially as you share my view on sending stuff out into things like the Cloud.

I recently installed Little Snitch and when I ran up CCC 4 to test it, Little Snitch told me that CCC wanted to connect to quite a few sites that as far as I could see had nothing at all to do with the running of the program. So I denied access and CCC seems to work just fine. I've asked the maker of CCC for comments regarding these seven sites that I have barred. be interesting to see their reply.

Do you run Little Snitch ?

[Dave2002]

Hi Anastasius

This is interesting about Google "phoning home" - I am getting more and more suspicious of Google's antics, though it's not the only company with some dubious nework behaviour.

Re the memory business - until I get the extra memory in my iMac I'm barely able to comment about memory pressure etc. Partly this is because the demands I now make on my computers are higher than they were 5 or more years ago, though that may not be the only reason. I remember about four years ago upgrading an earlier era iMac to increase the memory - that model was upgraded to 4 Gbytes, but because of the way the OS used the memory, it only had effectively 3 Gbytes after the upgrade. Even so, its performance beforehand was becoming completely unusable (I was not, by then the main user), and after the upgrade it was at least manageable.

The particular iMac I now want to use only has 4 Gbytes of memory - which is by modern standards pathetic. I have other machines with more memory, but that one has a decent screen, so I want to use it for video editing if I can. It is feasible to do this, but only just. I installed El Capitan, and have Adobe Premiere running on it. There is no doubt at all that if it is to function for this kind of work the memory will have to go up to a minimum of 8 Gbytes, or preferably even 16 Gbytes. I think the memory management system on El Capitan is similar to that in Mavericks. If I can get it upgraded to 16 Gbytes, then I could try out the notion of setting aside 8 Gbytes of RAM for a RAM drive, and seeing if that would actually go faster on some projects. I think it might, but there may be no point in doing that. Fairly obviously it would depend on what "files" would be written to/from the RAM drive, and also whether programs like Premiere use intermediate temporary files where there might be a potential speed advantage.

Possibly we are at the start of an era in which memory will get substantially cheaper, and we might see machines with 128 Gbytes of main memory or more, which would enable in memory editing even of Blu Ray videos, though 4k videos and even higher resolution videos may even then prove to be very challenging. I have noticed some iMacs with 32 Gbytes of memory.

I would expect it to take about 5 years for machines with 128 Gbytes or more of fast main memory to be readily available and perhaps affordable.

[Anastasius]

I'm beating my head against a brickwall here, I can see.

I'll leave you to your RAM disk.

Prize open your wallet and buy some more bloody memory. Then leave it alone and let Apple do its thing.



I also owe Carbon Copy Cloner an explanation. Apparently all those comments from Little Snitch are not quite 100% and are legit. CCC are very very kosher.

[Dave2002]

I'm beating my head against a brickwall here, I can see.

I'll leave you to your RAM disk.

Prize open your wallet and buy some more bloody memory. Then leave it alone and let Apple do its thing.



I also owe Carbon Copy Cloner an explanation. Apparently all those comments from Little Snitch are not quite 100% and are legit. CCC are very very kosher.

I'm sorry if I'm making your head hurt. I can neither agree nor disagree with you in my current state of knowledge, and with my current equipment and tests. This is because this machine does not have enough memory, something which I am about to rectify [I think Crucial can get extra memory to me by special delivery in a day or two]. Until recently lack of memory wasn't such a big issue, as I have other machines - some with more memory (e.g my MBP - though even that gets tight sometimes), and I wasn't trying to do video editing with a large screen.

Even if I tried to implement RAM disks to do a comparison against simply relying on Apple's own memory management system, it could not work on this machine with such a limited memory. It would only work if I could get enough memory into a system so that I could compare Apple with all memory - versus Apple with reduced memory allocated to RAM disc + RAM disc using most of the allocated reduced memory. Such an exercise may be pointless/fruitless on a machine even with 16 Gbytes of memory.

I can see from the rather neat Memory Monitor app that El Capitan has (at least) five categories of main memory, which are: Free, Wired, Cached, App and Compressed. You still like the Apple Activity Monitor, but in El Capitan I don't, because in order to run it I haven't found any way of getting a visual indication of the memory state which I can do with the Memory Monitor app which doesn't take up half of the display screen. In Mountain Lion I could set a visual icon in the Dock to show the Free memory, which was often useful.

Using the Memory Monitor app, if things get very slow, the Optimize button tries to create more Free memory. I don't know what algorithms it uses for that. It may try to reduce the App memory and the Wired memory preferentially before reducing the size of the Cache memory. It may also put some memory into the Compressed state.

It is also clear from the Memory Monitor app that although El Capitan clearly does a lot of things behind the scenes, it does not automatically optimise memory usage - at least not under all conditions. It is possible to set the preferences in Memory Monitor to do some automatic optimisation. I've not tried that yet as with the limited memory on this machine that could perhaps cause more problems that it would solve. What is clear, as with other earlier Mac OS X versions, is that as the available free memory decreases significantly, that a few of the application (probably the ones "in focus") slow down drastically. If working with a very tight allocation then the only sensible thing for me (at present) is to close down applications. This is the problem I hope to rectify by installing more memory - possibly even maxing it out on this machine which will go up to 16 Gbytes.

The article here - by James Coyle - on RAM disks in Linux - http://www.jamescoyle.net/how-to/599...-dd-and-bonnie
makes clear that it is quite hard to establish whether performance is improved on some (most?) modern OSs, because many of these now have optimised memory management, and use memory caches for file access. I suspect that there could still be a speed boost with RAM disks in some applications, but this might only show up on machines with rather more than 16 Gbytes of main memory. Use of RAM disks would give application writers more control, and that might help to speed things up, as they might be able to predict better how the memory/disk usage for the target application would go. There could also be other (small) benefits, as some other processing (e.g to decide whether to, or when, to write cache memory back to disk) would not cause any overhead, though that would be at the expense of losing any possibility of data recovery (for current work) if anything goes wrong.

Are you aware of any articles of claims by Apple about the way it does memory management within the OSs including and since Mavericks? Unless you are a developer or have an inside track, I don't think you can have a very complete picture of how Apple does that. Sometimes Apple is very coy about what it does - for example try to find out how it "really" does the management of its Fusion drives - though I do think the Fusion drives seem to work well enough.

I am curious as to why you are still using Mavericks - did you try Yosemite and/or El Capitan, and decide they didn't work for you? For my current purposes El Capitan seems good enough, and I can live with some annoyances where it doesn't work for me in as useful a way as earlier versions, but I won't get the full benefit until I've put the extra memory in on this machine.

[Anastasius]

I don't like the fonts, the screen layout in Mail, plus Mail didn't want to work for me without hassle plus a whole lot of other things. So since there was no 'Must Have' that I could see with El C I went back to Mavericks. I skipped Yosemite.

I give up. I've given you enough links to articles that explain the various types of memory and memory management in the latest OSX from Mavericks onwards. All that Memory Monitor does it make a nice user friendly fancy wraparound for the very basic purge command in terminal. That's all. You can Google purge and see why it isn't always a good idea since it defeats Apple's Memory Management to no good end. It defeats the whole ethos and objective of Apple's memory Management in Mavericks onwards. But your call. If you had more memory then you wouldn't be so much fixated on it.

Why don't you just hit F3 if you think you're running out of memory? Quick glimpse of the Memory Pressure and a simple Yes/No. Is it green ? Yes then stop worrying. F3 back to where you were. It really is that easy yet you seem intent in trying to micro manage what IMO Apple make an extremely good job of already.

I'm also convinced that with the complexity of the OS and various programs such as Mail that spending some time looking at the console logs might reveal a lot of repetitive and unnecessary use of CPU cycles (and memory).

[Dave2002]

Sorry, but you don't know my machine. I can tell you that without more memory it won't make any difference whether I hit F3 or any other fancy thing at present. Apple does not do a good job on my machine at the moment, though this is largely due to memory lack. Once I get the new memory I may well agree with you. Personally I have hardly any experience of Mvericks, though I think we have a couple of Mavericks based machines in the family. If I wanted to try that I could use a backup from one of these, or just simply try out the system on those machines. Mrs D has Yosemite on her laptop. It seems similar to EC, though I don't think it has the split screen features which are sometimes useful on the larger screen iMac.

I don't like the Spotlight on either of these recent OSs. With the old one (ML) there was usually an option to look at all the files in the Finder, and that was easy to configure to find files using the add on search criteria. The new versions are a waste of space, and one has to go into Finder by a circuitous route in order to get the same functionality. I am not interested in using Spotlight to search the web as well as local files.

One feature I've not explored or exploited is the file tagging, which may be an implementation of a feature I thought about over 40 years ago, when I worked for a computer company.

[Anastasius]

Ah, you just reminded me of another reason why I don't like EC...Spotlight. I agree with you totally.

Interestingly I just popped up to the Mac after lunch and noticed surprisingly that Memory Pressure was in the amber. I had a few tabs open in Safari and one or two websites must have been VERY memory hungry. Quitting Safari didn't free up as much memory as I would have liked and so a quick restart and the Mac is back to its sprightly self.

[Dave2002]

Ah, you just reminded me of another reason why I don't like EC...Spotlight. I agree with you totally.

Interestingly I just popped up to the Mac after lunch and noticed surprisingly that Memory Pressure was in the amber. I had a few tabs open in Safari and one or two websites must have been VERY memory hungry. Quitting Safari didn't free up as much memory as I would have liked and so a quick restart and the Mac is back to its sprightly self.

Is Spotlight better in Mavericks?

I prefer Mountain Lion and even Snow Leopard, but I have very limited experience of Mavericks. Spotlight in Yosemite seems similar to El C.

Just had a major panic trying to locate some video source files. Eventually (Phew!) found them on a memory card in one of my cameras. The clue was that I thought I had a large SDHC card, and I verified that by looking at my amazon orders. When I couldn't find the particular card I wondered where it might have got to ....

Fortunately looks as though the camera had not overwritten the vIdeo files, so copying these to a backup drive is now top priority. I think there is at least one copy somewhere else, but finding it/them using Spotlight hasn't worked out too well.

[Anastasius]

Yes, Mavericks has the 'proper' Spotlight.

Losing files is so easy. I had a folder of around 70GB of this and that...nothing that I would be totally upset at losing. It was on a drive that I wanted to use as a test bed for the remote CCC backup and it needed repartitioning and also to GUID. So I temporarily stored these over the local drive that I use as my SuperDuper backup. I just didn't think. Just whacked the folder over and went about my main objective. Totally forgetting that SD does an incremental backup to make the backup drive look identical to the drive in my iMac. Which did not have that 70GB folder. Ergo, SD did what it was supposed to do and deleted it!

I have a box of hard drives in my workshop and went through some of them and did find a fair chunk of the 70GB, but as I said, it was no great loss. I found several drives going back years and of low capacity and so they got trashed.